Google api quotas

[funderburkjim]

I started reading your book at learning.oreilly.com last year, and the Chapter 3 examples with Google poly api worked just fine.

In the meantime, I was able to get a faster broadband service (100mbs down),
and ran into a problem:
Namely, many of the images did not download (403 errors).

The following is a solution that may 'throttle down' the image retrieval. It seems to work.
Do you think it is reliable?
Is there a better solution to this kind of api quota issue?

[This is a revised renderResults function from example
http://localhost/webcomponentsinaction/chapter3/3.3.2-polysearchcomponent-attributes.html
]

   renderResults(assets) {
    // This code is slower, so quotas are NOT breached.
    // this sleep idea from https://flaviocopes.com/javascript-sleep/
   const sleep = (milliseconds) => {
    return new Promise(resolve => setTimeout(resolve, milliseconds))
   };
    for (let c = 0; c < assets.length; c++) {
     const img = document.createElement('img');
     img.src = assets[c].thumbnail.url;
     img.title = assets[c].displayName;
     img.width = "200";
     img.height = "150";
     sleep(50).then(() => {
      this.appendChild(img);
     });
    }
   }

[bengfarrell]

Hi there! This seems like a smart try! I'm very suspicious this is fixing your issue though. Unless I'm completely off-base, I think the fact that you're seeing this work is coincidental for whatever else is going on. I have a couple reasons why I'm led to believe this.

First off, check out the docs over at Poly: https://developers.google.com/poly/develop/api
You'll see that it says that you get 3000 request per minute with your API key, but "HTTP GET requests for resource files that make up an asset, such as a thumbnail or texture, do not count against this quota."

To me, this means that each search you do counts against that 3k requests per minute, but once you do that search, all of the images you get from the search are free.

So for you to hit that quota, you'd have to be type a search term and hit enter 3000 times in a minute to hit that!

Even if getting those thumbnails counted against you, it's only giving you 20 results per search (there's a "next page" token in the result JSON so you can do another request for the next set if you want more). So even IF images counted against you, you'd need to be running 150 searches per minute! That's still quite a bit.

Something is definitely suspicious for you to be hitting that quota - it makes me think something else is going on. Unfortunately I can't reproduce it on my end, things seem to just work well.

The other issue that makes me suspicious is that while your sleep code is pretty clever, it only does one thing: delay results from being added to the screen. Here's the problem: Your sleep code won't actually delay image loading!

The reason why, is that we have a for loop that iterates over all the images. Even with the sleep, the code isn't actually being delayed. Instead, the for loop is as speedy as it ever was, BUT right in the middle you set a timeout to add the image to the DOM. Having a timeout doesn't delay the execution of the for loop, it simply creates a timer off to the side that delays appendChild.

You can see this in action by putting something like console.log(c); at the end of the loop. And if you set the timeout to something crazy like 10000, you'll see that the for loop finishes instantly, and then 10 seconds later, all your images will make it to the DOM.

Another thing you might not realize is that images don't actually NEED to make it to the screen to load. Just setting img.src is enough to start loading that image. So, in the end, you're not actually throttling anything but the visual representation of what you're rendering.

I hope that helps illuminate whatever is really going on. I'm not quite sure what that is, but again, unless I'm mistaken, you had a great guess but I think might be going down the wrong path.

And BTW - I'm happy you're reading my book! Hope you're getting a lot out of it, but my publisher is Manning, so I hope you're not reading it at O'Reilly! :)

[funderburkjim]

Thanks for your comment. Based on your skepticism regarding the 'sleep' logic ,
I did some more experiments.
Results:

• sometimes the sleep logic also results in missing thumbnails

• I reverted to the github version of the 3.5.2 html, except for my api key, to be
  sure I didn't introduce any other problems.

• I've been using the web server in xampp for windows.
  Here's what the problem looks like, using Chrome browser and xampp server:
  

• Then I tried Firefox with xampp: similar results

• Then I tried the 'http-server' server you suggested from node.
  AND THE PROBLEM GOES AWAY!
  

Tried many different search terms, and all are fine with http-server.

I still wonder what might be wrong with xampp.

Anyway, thanks for your response.

[bengfarrell]

No prob! Yeah, this example in particular is fine even without a server (just load up the HTML from your local file system). If I had to guess, maybe there's some sort of weirdness with HTTP/HTTPS and cross origin loading. But even that doesn't sound right as it seems random which images it does or doesn't load.