Need help on Catalog feed generation from API and construct a file

[lakshmisaru]

Hi All,

We have a requirement to generate a file by calling different bulk api's and based on some rules we need to generate a file

ex: Call catalog api / inventory api and generate file with list of products which have inventory only.

What is the best practice to achieve this using GCP.

[imtaher]

To achieve this using GCP, you can follow the below steps:

1. Define your rules for generating the catalog feed file. This may involve specifying the criteria for selecting products or inventory items, and defining the format of the output file.

2. Use the Google Cloud Platform to call the appropriate bulk APIs to extract the data required for the catalog feed. You can use services like Cloud Functions or Cloud Run to execute code in response to an API request. You may need to authenticate your API calls using a service account or OAuth credentials.

3. Transform the raw data extracted from the APIs into the format required for the catalog feed file. You can use GCP services like Cloud Dataflow or Apache Beam to transform and manipulate the data.

4. Write the transformed data to a file in a format that can be consumed by downstream systems. You can use GCP services like Cloud Storage to store the output file.

5. Schedule the generation of the catalog feed file using a workflow management service like Cloud Composer. You can set up a workflow that calls the API, performs the required transformations, and writes the output file on a regular schedule.

Overall, the best approach will depend on the specific requirements of your use case. You may want to consult with a GCP expert or consultant to help design and implement a solution that meets your needs.

[lakshmisaru]

Thank you.