aws lambda

In Data Science, a productive model is an amazing work from you. You want to sell your excellent model to a Financial Business or a potential StartUp through the API subscription method. AWS Lambda and API Gateway will be the best choice for you. Deploying your model into production and scale-out them for thousands of thousands of users is almost done if it works on Serverless architecture. Doesn’t like deploying on an EC2, it costs you nothing if there is no usage.

在数据科学中，高效的模型是您的一项了不起的工作。 您想通过API订阅方法将出色的模型出售给金融业务或潜在的StartUp。 AWS Lambda和API Gateway将是您的最佳选择。 如果模型可以在无服务器架构上运行，那么几乎可以将模型部署到生产环境中并为成千上万的用户扩展模型。 不喜欢在EC2上进行部署，如果没有使用，则无需花费任何费用。

There are three popular frameworks using Python: SciKit-Learn, TensorFlow, and PyTorch. Today, TensorFlow is used widely for Deep Learning along with PyTorch while SciKit-learn is used for Machine Learning at most. In Mar 2019, Google has released a Tensorflow Lite version (TFLite). The lite version intends to use for Smart Phone and Embedded application. This is significant good news for Data Scientists.

有三种使用Python的流行框架：SciKit-Learn，TensorFlow和PyTorch。 如今，TensorFlow与PyTorch一起广泛用于深度学习，而SciKit-learn最多用于机器学习。 Google在2019年3月发布了Tensorflow Lite版本(TFLite)。 精简版打算用于智能手机和嵌入式应用程序。 对于数据科学家来说，这是一个重大的好消息。

It’s important to reduce the footprint of a Machine Learning or Neural Network model that runs in devices such as a Raspberry PI. When distributing it at client devices. It isn’t easy to protect your intellectual property from Jailbreaking. You need to keep your model on the server-side then exposing an API for any usage.

减少在Raspberry PI等设备中运行的机器学习或神经网络模型的占用空间非常重要。 在客户端设备上分发它时。 保护您的知识产权免遭越狱并不容易。 您需要将模型保留在服务器端，然后公开任何用途的API。

Does a full-version of Deep Learning frameworks is adaptive to modern software architecture such as Lambda serverless? No, it’s designed to support an end to end solutions for Data Science. Therefore, the full version of those frameworks is too large to fit in 250MB limited space of a Lambda function. To install the Tensorflow package, it takes nearly 1GB space!!!

完整版本的深度学习框架是否适应Lambda无服务器等现代软件架构？ 不，它旨在支持数据科学的端到端解决方案。 因此，这些框架的完整版本太大，无法容纳Lambda函数的250MB有限空间。 要安装Tensorflow软件包，需要占用近1GB的空间！！！

AWS has just released a Lambda EFS feature that allows you to attach an HDD-like to your Lambda. This will solve the problem of a very big size model file (over 250MB). To use an EFS, you must configure a lot of steps before being able to use it with Lambda.

AWS刚刚发布了Lambda EFS功能，可让您将类似HDD的内容附加到Lambda。 这将解决很大的模型文件(超过250MB)的问题。 要使用EFS，必须先配置许多步骤，然后才能将其与Lambda一起使用。

为什么我不能将TFLite与AWS Lambda一起使用？ (Why I cannot use TFLite with AWS Lambda?)

TFLite has a small footprint, so what is the troubling thing to use with Python running on a Lambda function? It’s all about the native builds for different processors, OS platforms, and customized kernels. AWS Lambda runs on its own, customized Linux so that, the pre-build TFLite from the community doesn’t work.

TFLite的占地面积很小，那么在Lambda函数上运行的Python所要使用的令人烦恼的事情是什么？ 所有这些都是针对不同处理器，OS平台和自定义内核的本机构建。 AWS Lambda在自己的定制Linux上运行，因此社区的预构建TFLite无法正常工作。

我该怎么做才能使其中之一起作用？ (What can I do to make one of them works?)

Let’s take TensorFlow to be the case. Cook it yourself to fit your needs! That’s all the stuff you have to do. But cook the Tensorflow Lite from SourceCode in the Amazon Linux platform resulting to have a compatible binary with AWS Lambda runtime. How to build a native library, cross-platform from SourceCode in my PC, this was a nightmare story when I was in the year of 2000. Now with Docker and the community, you are happy and very lucky. It is done easily within 15 minutes. Let’s create a Dockerfile:

让我们以TensorFlow为例。 自己做饭以满足您的需求！ 这就是您要做的所有事情。 但是，请使用Amazon Linux平台中的SourceCode烹制Tensorflow Lite，从而使其具有与AWS Lambda运行时兼容的二进制文件。 如何在我的PC上从SourceCode跨平台构建本机库，这是我2000年的噩梦。现在有了Docker和社区，您很高兴，也很幸运。 15分钟内即可轻松完成。 让我们创建一个Dockerfile：

FROM amazonlinuxWORKDIR /tfliteRUN yum groupinstall -y developmentRUN yum install -y python3.7RUN yum install -y python3-develRUN pip3 install numpy wheel pybind11RUN git clone --branch v2.3.0 https://github.com/tensorflow/tensorflow.gitRUN sh ./tensorflow/tensorflow/lite/tools/make/download_dependencies.shRUN sh ./tensorflow/tensorflow/lite/tools/pip_package/build_pip_package.shRUN pip3 install tensorflow/tensorflow/lite/tools/pip_package/gen/tflite_pip/python3/dist/tflite_runtime-2.3.0-cp37-cp37m-linux_x86_64.whlCMD tail -f /dev/null

All the things you need is to build a docker image that compiles the Tensorflow Lite library inside the amazonlinux image: (you can skip this step if you don’t have a Docker machine in your PC or don’t want to build this!)

您需要做的所有事情就是构建一个Docker映像，该映像可以编译amazonlinux映像内的Tensorflow Lite库:( 如果您的PC中没有Docker机器或不想构建它，则可以跳过此步骤！)

docker build -t tflite_amazonlinux .

This process will run in a couple of dozen minutes depending on your machine computation speed. There is a pre-built tflite_runtime and numpy library to be ready to use if you use simplify-cli to generate your Serverless project.

根据计算机的计算速度，此过程将在几十分钟内运行。 如果您使用simple -cli生成无服务器项目，则可以使用预建的tflite_runtime和numpy库。

适用于AWS Lambda的预构建TFLite库： (A Pre-built TFLite library for AWS Lambda:)

Simplify CLI offers you a tool to create a Serverless project, manage the deployment and its layers gracefully. Now, let’s create a Lambda function with “simplify-cli”.

Simplify CLI为您提供了一个工具来创建无服务器项目，并优雅地管理部署及其层。 现在，让我们使用“ simplify-cli”创建一个Lambda函数。

npm install -g simplify-cli # install serverless frameworkmkdir tensorflow-lite # create a project foldercd tensorflow-lite # enter this project foldersimplify-cli init -t python # generate a python project

In this default project, the file main.py will use the tflite_runtime library to load a pre-built model named detect_object.tflite that was generated before.

在此默认项目中，文件main.py将使用tflite_runtime库加载之前生成的名为detect_object.tflite的预构建模型。

In layer/python folder, there are two pre-built libraries for running on AWS Lambda:

在layer / python文件夹中，有两个预构建的库可在AWS Lambda上运行：

tflite_runtime (2.3.0)

tflite_runtime(2.3.0) numpy (1.19.1)

numpy的(1.19.1)

Everything you need to run your project is to setup two variables inside the .env file. To do so, you need an AWS Account with a Credential setup as a Profile or leave it blank if you use a default one.

运行项目所需的一切都是在.env文件中设置两个变量。 为此，您需要一个具有凭证设置的AWS账户作为配置文件，如果您使用默认账户，则将其留空。

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html

### - Application DeploymentDEPLOYMENT_ENV=demoDEPLOYMENT_BUCKET=tensorflow-deployment-37216DEPLOYMENT_REGION=eu-central-1DEPLOYMENT_ACCOUNT=your-aws-accountDEPLOYMENT_PROFILE=your-aws-profle### - Application StackNamePROJECT_NAME=TensorFlowTest### - Backend Serverless LambdaFUNCTION_NAME=detectObjectFUNCTION_RUNTIME=python3.7FUNCTION_HANDLER=main.handlerFUNCTION_SOURCE=src

The “37216” number you should change it to not having a conflicted bucket name with the other one who pickup this number before your test.

您应该将“ 37216”编号更改为与测试前使用该编号的另一个存储桶名称没有冲突。

Publishing your Python code to AWS Lambda service with this command:

使用以下命令将Python代码发布到AWS Lambda服务：

simplify-cli deploy

Then, deploy its layer that contains your TFLite library and numpy:

然后，部署其包含TFLite库和numpy的层：

simplify-cli deploy --layer --source layer

Finally, going to AWS Console, look for your Lambda function named as detectObject-demo then Test your code. You can see this link if you don’t know how to run a Test for your Lambda: https://aws.amazon.com/getting-started/hands-on/run-serverless-code/. You just need to do the “Step 4: Invoke Lambda Function and Verify Results”.

最后，转到AWS控制台，查找名为detectObject-demo的 Lambda函数，然后测试代码。 如果您不知道如何为Lambda运行测试，则可以看到此链接： https : //aws.amazon.com/getting-started/hands-on/run-serverless-code/ 。 您只需要执行“ 步骤4：调用Lambda函数并验证结果”。

(To continue with the Docker build, there is a script for your last step. That is the layer/build-tflite.sh. It will take out the result from Docker build to the layer folder.)

(要继续进行Docker构建，最后一步是一个脚本。这是layer / build-tflite.sh。它将从Docker构建中取出结果到layer文件夹。)

Testing with library loading time. The Lambda cold start took ~8 seconds. After the first request, it just takes around 25 milliseconds.

使用库加载时间进行测试。 Lambda冷启动耗时约8秒。 在第一个请求之后，只需要大约25毫秒。

Organize your Lambda with an API Gateway and AWS Marketplace. After finish this setup, you will be ready to sell your API to the world. Then, let’s start to find your customers.

使用API​​网关和AWS Marketplace来组织Lambda。 完成此设置后，您将可以将您的API推销给全世界。 然后，让我们开始寻找您的客户。

Leave your comment if you need a version of “tflite_runtime” for NodeJS on AWS Lambda.

如果您需要AWS Lambda上的NodeJS版本“ tflite_runtime”，请发表评论。

Follows my articles

跟随我的文章

A very simple Framework to manage your Infrastructure as codePublishing a TensorFlow Model on AWS Lambda to make a sell-able APIDIY — Build Yourself a Serverless Framework with 152 Lines of CodeThe On-Demand Wakeup Pattern to Overcome AWS Lambda Cold Start

一个非常简单的框架，以代码形式管理您的基础架构 在AWS Lambda上发布TensorFlow模型以制作可出售的API DIY —使用152行代码构建无服务器框架， 按需唤醒模式可克服AWS Lambda冷启动

翻译自: https://medium.com/problem-solving-blog/start-a-small-business-publishing-a-productive-model-on-aws-lambda-to-make-a-sell-able-api-77708490a759

aws lambda