WordPress To payload import

[bllfoad]

As an active user and advocate of PayloadCMS, I've been closely following its development and its growing impact in the web development community. Your dedication to providing a flexible and robust content management system is truly commendable .

I wanted to bring to your attention @jmikrut an opportunity that could significantly enhance the utility and appeal of PayloadCMS to a broader audience. Currently, WordPress holds a substantial market share, with approximately 40% of all websites worldwide built on its platform. This prevalence underscores the need for seamless migration options for users considering transitioning to alternative CMS platforms like PayloadCMS.

Given the significant number of websites potentially looking to make the switch, I believe there would be immense value in developing a dedicated plugin for PayloadCMS that facilitates the import of WordPress data. This plugin could streamline the migration process, making it more accessible and less daunting for users who are interested in harnessing the power of PayloadCMS.

[DanRibbens]

Hey @bllfoad,

This is a good topic! We've been discussing the idea of maintaining migration guides from different platforms in the Payload website docs.

With any migration there are two steps:

1. A developer needs to put together the Payload project containing all the fields in collections and globals that are needed.
2. Data and file migrations. Pulling all the data out of an existing system to be inserted into the Payload project is scriptable but requires custom development also.

We are looking at building an import/export plugin that assists with all ETL workflows to avoid having to write custom scripts which is the currently recommended approach, this helps with the 2nd item.

I don't think it would possible to write a script for the first since there are so many plugins and different versions and configurations that WordPress can have. For this reason I see it as more likely that we publish a guide than actual scripts aside from possibly some helper utilities that can either exist as import scripts, plugins or built into the core of Payload.

I'm interested in your feedback and wonder if you have some other thoughts on how to more easily provide migration support in a way that is efficient with our own limited resources.

[bllfoad]

Hey @DanRibbens ,
First PayloadCMS is awesome second :
Thanks for your detailed outline on the potential for migration guides and the development of import/export plugins. I completely understand the challenges involved, especially considering the diversity of configurations in systems like WordPress.

Considering our discussions and your plans, I propose a targeted approach for a WordPress to PayloadCMS migration plugin. The idea is to focus on simplifying migrations by specifically supporting key data types such as posts, categories, users, and tags. This plugin would function by extracting data from WordPress through XML exports—this is a common format that WordPress can generate natively or through various plugins.

Initially, the plugin would not handle posts with custom fields or more complex data types. This would streamline the initial development and ensure robust handling of the standard data types. Over time, we can evaluate the possibility of expanding the plugin’s capabilities to include more complex data scenarios, depending on user feedback and the evolution of our core functionalities.

This approach would offer a standardized migration path, greatly reducing the need for custom scripts and providing a practical demonstration for the migration documentation on the Payload website. It could also serve as a foundation for broader ETL support within PayloadCMS.

I’m eager to hear your thoughts on this proposal and discuss how we might collaborate to develop this concept further.

[andershermansen]

I have written a migration script to move products from a woocommerce solution into payload.

I used woocommerce rest api to read and payload rest API to write. Most of the program was about data adjustment for how we wanted it saved in payload.

If I had an import/export plugin to help me with this that would be nice.

[wkentdag]

I just posted an outline of my migration script in the community help discord. It's very tailored to my specific WP configuration so I just described my general approach, but I'd be happy to share code too once I clean the script up a bit.

I don't think it would possible to write a script for the first since there are so many plugins and different versions and configurations that WordPress can have. For this reason I see it as more likely that we publish a guide than actual scripts aside from possibly some helper utilities that can either exist as import scripts, plugins or built into the core of Payload.

^ I agree with that, although a bulk payload.create would be nice!

[GoldinGuy]

Link is broken for me – could you share again? Interested in your approach.

[wkentdag]

you made need to join the discord to view the link (it hasn't changed). But here's what I wrote:

i am in the middle of migrating a large wordpress project to payload. my migration script is pretty heavily tailored to my specific application, but here's a high level outline of my approach:

1. download a SQL dump of the wordpress database and store locally. this is my source of truth for the existing documents. this approach reduces latency, and cpu load on the existing (live) wordpress app, but requires you to "freeze" publishing on wordpress while the migration is in process.
2. migrate the "primary" content types, without relationships. for example in my application: books, authors, and publishers.
3. circle back and migrate the relationships: for example, assign author(s) and publisher to books.
4. (optional) delete any "orphaned" content, eg authors or publishers with 0 book relationships.
   a couple notes...

• every payload collection has a wordpress_id readonly, indexed field with this format: ${post_type}:${ID}. this makes step 3 easy: query a book's authors by their wordpress_ids, retrieve their payload IDs, use those to build the cross-collection relationships.
• For uploads, I have temporarily disabled filesRequiredOnCreate. This way I can create a document for every upload in my db without uploading the file. File upload would be the final step (5), only after I've built relationships between posts and media uploads in payload, and weeded out any orphaned entries (we have 100s of gbs of uploads, most of which are garbage). once the migration's finished, I'll re-enable filesRequiredOnCreate.
• for step 2, I iterate over every row returned from wordpress and process them in a queue (better-queue). the queue helps me manage throughput, handle exceptions, etc.

fyr, an old discussion about importing from csv: https://payloadcms.com/community-help/github/user-csv-import-for-a-collection

and also heads up, I wasn't able to replicate wordpress's media URL structure with payload. more context here: #1153 (comment)