Python3 is a resource hog compared to Python2.7 - Can't maintain costs, any solutions?

[Kaan]

My Python2.7 instances were 3.4mb's - now the Python3 instance is 200mb - it's apples vs oranges in terms of disk size, memory, but my issue is:

On Python2.7 F2 instance was performing well, On Python3 - F2 instances die in minutes and the game is unplayable, even the F4 instance is spotty and my game's performance is low, costs are almost 2X compared to F2 - and on F4_1G - which performance is acceptable at, costs are almost 5X

Overall App Engine costs are so high that even at F2 - I was 1:1 in terms of income/costs, at F4 I'm burning from pocket

Code is open source, you can see it at: https://github.com/kaansoral/adventureland

My requirements.txt is:

Flask
lxml
appengine-python-standard

Is there a gotcha, a common solution that I missed by chance? Or is it time to move away 😞

[daskalou]

Have you tried tweaking the app.yaml config values such as the gunicorn entrypoint, or the scaling concurrent requests settings, to see if these cause less instances of your app to be created?

[Kaan]

Thanks for your reply

I actually force a single instance via:

runtime: python39
app_engine_apis: true
builtins:
- deferred: on

instance_class: F4
automatic_scaling:
min_idle_instances: 0
max_idle_instances: 1
min_pending_latency: 14000ms

First time I'm hearing about gunicorn, is it a different approach to serving instances or is it the new way behind the scenes that I can configure and make a change as well?

[daskalou]

To force a maximum number of instances, you need to use:

automatic_scaling -> max_instances

and not:

automatic_scaling -> max_idle_instances

Regarding the entrypoint, something like this works well for us:

entrypoint: gunicorn --bind :$PORT --workers 2 --worker-class=gthread --threads=<whatever our automatic_scaling -> max_concurrent_requests value is, divided by 2> --preload

In order to get the above gunicorn command to work, I believe you also need to add this to your requirements.txt:

gunicorn[gthread]

[NoCommandLine]

Hi,

Take a look at this previous discussion and this one too, about moving from Python 2 to Python 3 and see if the tips on reducing App size and controlling costs helps.

   ......NoCommandLine ......
https://nocommandline.com

        Analytics & GUI for 
App Engine & Datastore Emulator

[micdearmas]

For what is worth, after migrating my apps to Python3 I am spending a little bit less in cloud resources in general and I was already spending very little when using Python27. That was the case when I had them in Python39 and it is still the case now that I have most of them in Python311.

Having said that, as far as I can tell, I don't use any resources other than the defaults. That is, I don't specify the type of instance, so the use instance_class: F1.

In the past when I got spikes in resource consumption, I played a little bit with settings min_idle_instances, max_idle_instances, min_pending_latency, max_pending_latency, max_instances but when things settled down I could go back to defaults for most cases.

I've never set instance_class for any of my apps and I think it has been possible because of the way I've architected the apps:

- rely on server sessions (in my case would be flask sessions now) as little as possible
- whatever you need to keep for each user (or for the app in general) store it in Datastore
- before reading from Datastore, check if you already have that item in memcache and get it from there
- after reading an item from Datastore put in memcache
- if you end up having in Datastore entities that you no longer need, you can have scheduled processes that do the cleanup once or twice a day or as often as needed, also have used the Time-to-Live(TTL) feature that they introduced some time ago

That is an approach that has worked great for me and I think it could be used for most of the applications I've come across.

[Kaan]

Thank you for all the suggestions!

After the "entrypoint" change, I was even able to get back to F1 and as far as I monitored for the last 15 minutes, it's been stable, the one instance is live, memory creeps up but goes down later on, even 15 minutes is good though

Growing up with App Engine, I like to think I designed my app simple

Just went back to "Time-to-Live(TTL)" - but got hit with the 400: TTL feature currently doesn't work with concurrency mode 'Optimistic With Entity Groups'. Please change it before using TTL. https://cloud.google.com/datastore/docs/upgrade-to-firestore#optimistic_with_entity_groups_concurren... Restriction and I don't see a way around it

It seems when it first came around I added an "expire_at" property to applicable stuff, but I recall another issue at the time