Click here to close this tab and return to the app.

Automated Feed Website Crawl vs File Upload in Google Merchant Center

Oct 2, 2023

Google's Automated Feeds Blog Post Header Image

In this blog post, we compare Google's Automated Feeds to File Uploads for submitting product data feeds to Google Merchant Center.

Advantages of Google's Automated Feeds

Wondering if the new, free Automated Feed feature by Google is worthwhile? Let's dig in to find out.

The primary advantage of Google's Automated Feed is that you only have to provide your website URL; no data feed file preparation needed. Google's web crawler, Googlebot, scans through your website, automatically finds your products, and adds them to your Products tab in Google Merchant Center. Furthermore, the website crawl is performed on a regular, automated basis to keep track of changes to pricing and availability.

Disadvantages of Google's Automated Feeds

Unfortunately, the website crawl requires 'structured data' and 'sitemap' information in your website in order to function. Structured data, or schema.org markup, is product data in the HTML source code of your product pages that is hidden to site visitors, but visible to web crawlers. Here is an example:

<div itemtype="https://schema.org/Product" itemscope> <meta itemprop="mpn" content="Acme-123" /> <meta itemprop="name" content="Amazing Widget Winder" /> <link itemprop="image" href="https://example.com/photos/16x9/photo.jpg" /> <meta itemprop="description" content="Use this amazing widget winder to wind any widget." /> <div itemprop="offers" itemtype="https://schema.org/AggregateOffer" itemscope> <meta itemprop="lowPrice" content="119.99" /> <meta itemprop="priceCurrency" content="USD" /> </div> <div itemprop="brand" itemtype="https://schema.org/Brand" itemscope> <meta itemprop="name" content="Acme" /> </div> </div>

Many shopping cart providers (like Shopify and BigCommerce) include these tags out of the box. However, if you have a proprietary shopping cart you will have to hire a developer to add the tags. Although many shopping cart providers already include the tags, their implementation may be missing many key pieces of information or they may put incorrect data in the tags.

For stores that sell products with variations (e.g. clothing or shoes by size and color), implementations of structured data are especially poor. The implementation requires the correct image, size, color, price, and availability for each variation. Structured data is difficult to implement for products with variations, because data for multiple variants must be correctly specified on one page.

As a general rule, shopping cart providers do not focus on data feed quality, and therefore their implementations of structured data tags are often buggy and lacking in many ways. It may be impossible to get buggy structured data implementations fixed, as they will be a low priority to the shopping cart provider. It may further be difficult or impossible to get the structured data tags customized in any way for your particular store.

In our experience, even for merchants on top-tier e-commerce cart like BigCommerce and Shopify, which have strong implementations of structured data, the website crawl is buggy and creates unnecessary diagnostic errors and warnings. Google Merchant Center recently started showing a message saying:

Google can automatically add your products from your website.
These 12345 products will automatically be added to Merchant Center from October 23, 2023. You can also start adding them now so that you can appear more often in search results.

However, once enabled, Google imports invalid products, with many critical data errors like invalid identifiers, incorrect price/availability, and duplicate listings. On average, we are seeing about 10% of the items with errors in our clients who had this setting enabled, with the data quality of the rest of the items unclear. We are disabling it as part of our Feed Optimization Service.

Advantages and Disadvantages of a File Upload Data Feed

The free website crawl offered by Google may be a good option for merchants who need a quick way to get most of their listings onto Google Shopping. For more established merchants, a data feed file prepared and uploaded to Google may be a better choice. The main disadvantage is that this requires a monthly fee paid to the data feed provider (like us) to maintain and submit the feed to Google.

For this fee, a purpose-built data feed file is generated to the exact specifications of Google, with invalid items filtered out and tight control over data quality. There is little to no up-front fee, since you are subscribing to a software as a service. Any up-front investment you wish to make can instead be focused on content improvements in your product catalog, e.g. adding missing images, inputting GTIN/UPCs, setting product types, etc. That investment goes into your catalog itself, keeping the benefits in-house and generating side benefits for organic SEO. This may be a much smarter investment than implementing structured data in your store, which is essentially data feed software built-into your store that will require costly maintenance over time.

In order to do a File Upload, Merchant's need to first subscribe to the data feed service by registering an account. Then, they need to add a feed either by following the setup guide or purchasing the setup and optimization service. The data feed service runs on a Daily basis to submit an up-to-date data feed file to Google. So every day, it imports all the latest, raw product data from your store catalog via the API. Using the raw product data, a tailored data feed file is generated, and finally uploaded to Google. In addition to the Daily updates, the service includes monitoring of the uploads, updating the feed to the latest feed specifications updates, and technical support for errors/warnings you may encounter in Google Merchant Center.

Conclusion

In summary, Google's Automated Feed is a new, free service that creates your product listings via a website crawl. Because of its requirements for structured data, which is poorly supported by most shopping carts, the resulting listings may have many data quality issues, errors, and warnings. Instead, for a modest monthly fee, a daily file upload provided by a data feed service can provide higher data quality and better support, without any up-front investment costs.

Next Steps

More Blog Posts