-
Notifications
You must be signed in to change notification settings - Fork 6.6k
Add FastAPI Workers and Workers AI tutorial #20653
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: production
Are you sure you want to change the base?
Add FastAPI Workers and Workers AI tutorial #20653
Conversation
Howdy and thanks for contributing to our repo. We review new, external PRs within two (2) weeks. If it's been two weeks or longer without any movement, please tag the PR Assignees in a comment. |
src/content/docs/workers-ai/tutorials/create-fastapi-python-worker-and-workers-ai.mdx
Outdated
Show resolved
Hide resolved
src/content/docs/workers-ai/tutorials/create-fastapi-python-worker-and-workers-ai.mdx
Outdated
Show resolved
Hide resolved
src/content/docs/workers-ai/tutorials/create-fastapi-python-worker-and-workers-ai.mdx
Outdated
Show resolved
Hide resolved
src/content/docs/workers-ai/tutorials/create-fastapi-python-worker-and-workers-ai.mdx
Outdated
Show resolved
Hide resolved
src/content/docs/workers-ai/tutorials/create-fastapi-python-worker-and-workers-ai.mdx
Outdated
Show resolved
Hide resolved
Howdy and thanks for contributing to our repo. The Cloudflare team reviews new, external PRs within two (2) weeks. If it's been two weeks or longer without any movement, please tag the PR Assignees in a comment. We review internal PRs within 1 week. If it's something urgent or has been sitting without a comment, start a thread in the Developer Docs space internally. PR Change SummaryAdded a tutorial for creating a FastAPI application using Cloudflare Workers and integrating Workers AI for generating alternate endings for shows and movies.
Added Files
How can I customize these reviews?Check out the Hyperlint AI Reviewer docs for more information on how to customize the review. If you just want to ignore it on this PR, you can add the Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add What is Hyperlint?Hyperlint is an AI agent that helps you write, edit, and maintain your documentation. Learn more about the Hyperlint AI reviewer and the checks that we can run on your documentation. |
Co-authored-by: hyperlint-ai[bot] <154288675+hyperlint-ai[bot]@users.noreply.github.com>
return {"message": "Hello World"} | ||
``` | ||
|
||
Inside on_fetch, asgi.fetch(app, request, env) processes the incoming request using your FastAPI app. The asgi module facilitates the communication between the Worker environment and your ASGI application. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit:
Inside on_fetch, asgi.fetch(app, request, env) processes the incoming request using your FastAPI app. The asgi module facilitates the communication between the Worker environment and your ASGI application. | |
Inside `on_fetch`, `asgi.fetch(app, request, env)` processes the incoming request using your FastAPI app. The asgi module facilitates the communication between the Worker environment and your ASGI application. |
|
||
### 1.1 FastAPI Code | ||
|
||
Create the main application file (main.py) in the src folder: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe:
Create the main application file (main.py) in the src folder: | |
Make a `src` folder and create the worker entrypoint `src/main.py` with the following contents: |
|
||
### 1.3 Configure the Cloudflare Worker (wrangler.toml): | ||
|
||
Create a new file wrangler.toml at the root and paste the content below in that file. This file configures how your Cloudflare Worker is built and deployed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe:
Create a new file wrangler.toml at the root and paste the content below in that file. This file configures how your Cloudflare Worker is built and deployed. | |
Create a new file `wrangler.toml` at the root. This file configures how your Cloudflare Worker is built and deployed. Paste the following contents in that file. |
name = 'what-if-ai-backend' | ||
main = "src/main.py" | ||
compatibility_flags = ["python_workers"] | ||
compatibility_date = "2023-12-18" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we use a more recent date?
```python | ||
env = req.scope['env'] | ||
CLOUDFLARE_API_KEY = env.CLOUDFLARE_API_KEY | ||
CLOUDFLARE_ACCOUNT_ID = env.CLOUDFLARE_ACCOUNT_ID | ||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we be using a binding for this? I think we shouldn't have to manipulate api keys at runtime. @dom96 do you know what the best practices here would look like?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where does req.scope['env']
get its env vars from?
The best practices here are to use Secrets: https://developers.cloudflare.com/workers/configuration/secrets/.
I think the best option may be to use the importable env:
from workers import import_from_javascript
cloudflare_workers = import_from_javascript("cloudflare:workers")
cloudflare_workers.env.CLOUDFLARE_API_KEY
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be equivalent, req.scope['env']
is the real env. So I guess that's a +1 for this approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But importable env would work well too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @hoodmane and @dom96 for the thoughtful feedback. I considered using bindings for this tutorial and even discussed it with @mikenomitch, who kindly provided a code snippet showing how it could be implemented.
messages = pyodide.ffi.to_js(
[
{ "role": "system", "content": "You are a funny and creative joke teller." },
{ "role": "user", "content": "Tell a joke about Cloudflare." },
],
dict_converter=js.Object.fromEntries, # Note: To use messages via javascript API, we must convert messages to array of objects.
)
job = {
"messages": messages,
"seed": 1739384258,
"max_tokens": 4096,
"temperature": 0.7,
"top_p": 0.7,
"repetition_penalty": 1.0,
"frequency_penalty": 0.0,
"presence_penalty": 0.0
}
resp = await env.AI.run(
'@cf/meta/llama-3.1-8b-instruct-fast',
**job,
)
resp_json = resp.to_py() # JsProxy -> dict
js.console.log('messages_response =', json.dumps(resp_json, indent=4, default=str))
For this tutorial, I would prefer to keep the tutorial without bindings. My main priority is ensuring the content remains beginner-friendly and accessible to newcomers who might be overwhelmed by additional complexity.
Once users grasp the fundamentals presented in this tutorial, they'll be better positioned to explore more advanced concepts like bindings in the upcoming tutorial in this series.
What's your take on this? Do you think bindings are essential even for beginners in this context?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would say that using httpx for this is more complex than the binding approach. It's also less secure since you have to manage the API keys in the code, whereas the bindings manage that for you.
The HTTP API for AI in Cloudflare is for those that cannot use Workers, so we should be showing bindings when we do use Workers wherever possible.
Thanks @HackyRoot. It looks generally reasonable to me. We should try to do some copy editing on it. My main concern from a technical perspective is the approach to workers-ai. I think this should be ideally done via a binding. Hopefully we can get someone who is more familiar than I am with how bindings work to explain what we should be doing here. |
Summary
Add a tutorial for FastAPI Python worker and Workes AI.
Documentation checklist