Deploying Mozilla DeepSpeech models to AWS Lambda using Serverless

Intro

Setting up Serverless

$ sudo apt-get install nodejs$ sudo apt-get install npm
npm install -g serverless
$ serverless --version
$ serverless login

Installing Deepspeech

$ conda create -n deepspeech_lambda python=3.6
$ conda activate deepspeech_lambda
$ pip install deepspeech
$ pip install scipy

Creating a Serverless Project

$ mkdir deepspeech_lambda && cd deepspeech_lambda
$ serverless create --template aws-python
import jsondef inferHandler(event, context):
body = {
“message”: “Go Serverless v1.0! Your function executed successfully!”,
“input”: event
}
response = {
“statusCode”: 200,
“body”: json.dumps(body)
}
return response
service: deepspeech-lambda-demo
app: deepspeechlambda
org: lukasgrasse
provider:
name: aws
runtime: python3.6
stage: dev
region: us-east-1
functions:
infer:
handler: infer.inferHandler
timeout: 30
events:
- http:
path: infer
method: post
$ serverless deploy -v
$ curl -X POST https://<some id>.execute-api.us-east-1.amazonaws.com/dev/infer

Adding DeepSpeech to the Serverless Project

pip freeze > requirements.txt
serverless plugin install -n serverless-python-requirements
custom:
pythonRequirements:
dockerizePip: true
slim: true
zip: true
├── infer.py├── model│   ├── alphabet.txt│   └── output_graph.pbmm├── package-lock.json├── package.json├── requirements.txt└── serverless.yml

Updating the Handler Function

try:
import unzip_requirements
except ImportError:
pass
from deepspeech import Model, printVersions
import json
import base64
import io
import numpy as np
import scipy
import scipy.io.wavfile
SAMPLE_RATE = 16000
BEAM_WIDTH = 500
N_FEATURES = 26
N_CONTEXT = 9
ds = Model('model/output_graph.pbmm' , N_FEATURES, N_CONTEXT, 'model/alphabet.txt' , BEAM_WIDTH)def inferHandler(event, context):
body = json.loads(event['body'])
content = base64.b64decode(body['content'])bytes = io.BytesIO(content)samplerate, data = scipy.io.wavfile.read(bytes)recognized_text = ds.stt(data, samplerate)

response = {
"statusCode": 200,
"body": recognized_text
}

return response
$ serverless deploy -v
(echo -n '{"content": "'; base64 test.wav; echo '"}') | curl -H "Content-Type: application/json" -d @- https://<some id>.execute-api.us-east-1.amazonaws.com/dev/infer

Conclusion

Get in Contact

References

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Lukas Grasse

CTO and Co-Founder of Reverb Robotics Inc | Machine Learning and AI Consultant | Ph.D. Student in Neuroscience @ U of L.