[null,null,[],[[["\u003cp\u003eGooglebot has a 15MB size limit for processing the initial HTML content of a webpage, although it fetches referenced resources like images and videos separately.\u003c/p\u003e\n"],["\u003cp\u003eThis limit has been in effect for a long time but was recently documented to help web developers with debugging.\u003c/p\u003e\n"],["\u003cp\u003eContent exceeding the 15MB limit is disregarded by Googlebot during indexing, impacting only a small percentage of webpages.\u003c/p\u003e\n"],["\u003cp\u003eWeb developers are encouraged to optimize large HTML files by moving inline scripts and CSS to external files.\u003c/p\u003e\n"],["\u003cp\u003eYou can use browser developer tools or command-line tools like cURL to check the size of your webpage.\u003c/p\u003e\n"]]],["Googlebot only indexes the first 15 MB of fetched content for specific file types, a limit that's long-standing but recently documented. This limit applies to the initial request's bytes, including subresources like CSS and JavaScript. Content exceeding 15 MB is dropped. Most web pages are significantly smaller than this threshold. Data URIs contribute to the file size. Users can check page sizes via browser developer tools or cURL. Googlebot accesses images and videos via URLs separately.\n"],null,["# Googlebot and the 15 MB thing\n\nTuesday, June 28, 2022\n\n\nOver the last few days we've received a great deal of questions about a recent update to\n[our documentation about Googlebot](/search/docs/crawling-indexing/googlebot).\nNamely, we've documented that Googlebot only ever \"sees\" the first 15\n[megabytes](https://en.wikipedia.org/wiki/Megabyte#-_-) (MB)\nwhen fetching\n[certain file types](/search/docs/crawling-indexing/indexable-file-types).\nThis threshold is not new; it's been around for many years. We just added it to our documentation\nbecause it might be helpful for some folks when debugging, and because it rarely ever changes.\n\n\nThis limit only applies to the\n[bytes](https://en.wikipedia.org/wiki/Byte#;)) (content)\nreceived for the initial request Googlebot makes, not the referenced resources within the page.\n| **March 16, 2023**: To further clarify, each individual subresource fetch (in particular CSS and JavaScript) is bound to the 15MB limit.\n\n\nFor example, when you open `https://example.com/puppies.html`, your browser will\ninitially download the bytes of the HTML file, and based on those bytes it might make further\nrequests for external JavaScript, images, or whatever else is referenced with a URL in the HTML.\nGooglebot does the same thing.\n\n\n**What does this 15 MB limit mean to me?** \n\nMost likely nothing. There are\n[very few pages](https://twitter.com/paulcalvano/status/1541402096897069056)\non the internet that are bigger in size. You, dear reader, are unlikely to be the owner of one,\nsince the\n[median size of a HTML file is about 500 times smaller](https://httparchive.org/reports/page-weight#bytesHtml):\n30 [kilobytes (kB)](https://en.m.wikipedia.org/wiki/Kilobyte).\nHowever, if you are the owner of an HTML page that's over 15 MB, perhaps you could at least move\nsome inline scripts and CSS dust to external files, pretty please.\n\n\n**What happens to the content after 15 MB?** \n\nThe content after the first 15 MB is dropped by Googlebot, and only the first 15 MB gets forwarded\nto indexing.\n\n\n**What content types does the 15 MB limit apply to?** \n\nThe 15 MB limit applies to fetches made by Googlebot (Googlebot Smartphone and Googlebot Desktop)\nwhen fetching\n[file types supported by Google Search](/search/docs/crawling-indexing/indexable-file-types).\n\n\n**Does this mean Googlebot doesn't see my image or video?** \n\nNo. Googlebot fetches videos and images that are referenced in the HTML with a URL (for example,\n`\u003cimg src=\"https://example.com/images/puppy.jpg\" alt=\"cute puppy looking very disappointed\" /\u003e`\nseparately with consecutive fetches.\n\n\n**Do data URIs add to the HTML file size?** \n\nYes. Using\n[data URIs](https://en.wikipedia.org/wiki/Data_URI_scheme)\nwill contribute to the HTML file size since they are in the HTML file.\n\n\n**How can I look up the size of a page?** \n\nThere are a number of ways, but the easiest is probably using your own browser and its Developer\nTools. Load the page as you normally would, then launch the Developer Tools and switch to the\nNetwork tab. Reload the page, and you should see all the requests your browser had to make to\nrender the page. The top request is what you're looking for, with the byte size of the page in\nthe Size column.\n\n\nFor example, in the\n[Chrome Developer Tools](https://developer.chrome.com/docs/devtools)\nmight look something like this, with 150 kB in the size column:\n\n\nIf you're more adventurous, you can use [cURL](https://curl.se/)\nfrom a command line: \n\n```\ncurl \\\n-A \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36\" \\\n-so /dev/null https://example.com/puppies.html -w '%{size_download}'\n```\n\n\nIf you have more questions, you can find us on\n[Twitter](https://twitter.com/googlesearchc)\nand in the\n[Search Central Forums](https://support.google.com/webmasters/community),\nand if you need more clarification about our documentation, leave us feedback on the pages\nthemselves.\n\nPosted by [Gary Illyes](https://garyillyes.com/+)"]]
