Need help with "gcloud builds submit" command

[SurajT]

Hi guys, 

I am trying to build my Dockerfile using the following command for a Python project: "gcloud builds submit --tag gcr.io/287924944473/container-name" 

It starts and begins downloading and starts the pip install -r requirements step I have outlined in my Dockerfile however one of the packages (torch+cuda) is 1.8GB and the build keeps crashing. I have tried to increase my instance size from 4GB memory to 128GB memory (everything inbetween) but I still get the following 137 Error: 

Killed
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 137
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 137

Could someone please help me with this? I have tried to increase the instance memory size to no avail. I am not sure where to go next. My suspicion is that the "gcloud builds submit" command imposes a memory limit on the build itself? But I am really unsure as I am new to GCP...

[kolban]

Are you able to build your image on a Compute Engine?   If I were in your shoes, I'd created a Container Optimized OS based Compute Engine and try and run your Docker build step manually there.  If it succeeds, that's one thing, however if it fails with the out of memory issue then now we have eliminated Cloud Build and can focus on how to solve using just Google Cloud and Compute Engines.

[SurajT]

Hi Kolban - thanks for your reply. I have managed to create a Container Optimised OS however I'm running into a problem:

Previously, I was cloning my entire repository from Cloud Source Repositories to the instance using "gcloud source repos clone [REPO NAME] --project=[PROJECT-ID]" and prior to this I was enabling git in the instance using "sudo apt-get git" however it seems the Container Optimized OS has neither apt-get nor gcloud commands. 

Given that I have a repository with a Dockerfile in it - how can I access the repo and begin the build using the Container Optimized OS?

[kolban]

Howdy SurajT, ... well ... <sigh> ... that's taught me a lesson.  I'm going to assume then that the Container Optimized OS is great for running Docker containers but not great at all for building an image.    My goal was to have a recipe where we could use Google Cloud Compute Engines to try and build your image and try it under a variety of permutations. I'm still thinking that is the way to go ... but there is going to be effort to get there.  What I'd be tempted to do now is create a Debian Linux Compute Engine instance.  I know that will have Google Cloud SDK installed (gcloud) and will also have apt-get installed.  We would then have to install Docker CE.  Now we would have an environment to test ... and now the experiments can start.   However, as you see, no immediate answer.