Did you know that the combination of all US stock markets has over 10,000 different stocks you can trade?
As amazing as that is, if you're building your own stock trading bots, it may present quite a challenge for you! You might be thinking:
- How on earth do I get access to all that historical market data for my trading algorithms?
- How can I analyze market data at that kind of scale?
- How hard will it be to get access to that historical price data?
When I first started stock trading at the age of 10, the answer to those questions was "very hard, expensive, and time consuming". Today we would call it manual trading, and my process was:
- Purchase a national news paper
- Go to the business section
- Find the listing of the previous days opening, high, low and close data
- Note it down in my notebook
- Manually analyze my data
Clearly that's not a scalable process!
I'm glad to say that in todays computer driven world, things are much easier, and I'm going to show you how!
What You'll Learn Through My Blog
I'm going to show you everything you need to build your own trading bot from scratch, including:
- Your own completely customizable development environment, including the worlds most popular technical indicators, and cutting-edge AI from ChatGPT
- The ability to retrieve historical stock data on over 10,000 US based stocks
- The power to calculate stock technical indicators on any stock you retrieve
- The insight to manage your risk
The premium episodes expand this amazing knowledge base to include:
- A whole host of technical indicators
- Advanced risk management
- Advanced strategy building
What You'll Have by the End of this Episode
By the end of this episode, you'll be able to retrieve historical market data for your stock trading bot for over 10,000 different stocks. You'll have learned how to:
- Connect to the Alpaca.Markets API
- Query a stock of your choice
- Format the data for use in your trading strategies
- Use Python code to retrieve the above
By the way, if you want to check out the completed code for this episode, here's a link to my something-cool repo on GitHub. Don't forget to drop me a star to say hi ❤️
What You Need to Have Completed
The best and easiest way to complete this episode is to have a development environment set up in the same way as my "Build Your Own Trading Bot Dev Environment that works from Anywhere and includes TA Lib, AI". You can check out that free episode using the button below.
Helpful Help from TradeOxy
TradeOxy's mission is to democratize algorithmic trading, and that mission extends to my blog.
If at any point you get stuck in this series, then reach out to us on our Discord chat. I and my team will be more than happy to help out (just make sure it's about our content).
What is Alpaca Markets?
In their own words, Alpaca Markets is an API for Stock and Crypto Trading. They provide a full range of features for everything from retrieving historical data, making trades, and keeping an eye on your portfolio.
Why do I use Alpaca Markets for my Automated Trading Bots?
Compared so some of the other platforms I've used and developed for, I've found Alpaca Markets extremely easy to use, and really easy to integrate into my automated trading bots. They're also very active in adding new features and asset classes to their API, covering:
Specifically for our purposes, Alpaca Markets offers some great advantages:
- API is pretty well documented
- API is easy to use
- Provides a generous free tier
Note. These words are my own, I do NOT receive a commission for this!
How Do I Sign Up for Alpaca Markets?
To sign up for Alpaca Markets (required for the rest of this episode), follow the link in the button below.
Connect to Alpaca Markets
Now that you've signed up to Alpaca, lets get your trading bot connected.
Alpaca Markets Interaction File
Assuming you're using the development environment we created in this episode, head to your file system, and create a file called
Here's what your full file structure should look like:
This file is going to contain everything you need to interact with the Alpaca Markets API.
Alpaca Markets Authentication: Get Your Keys
Authenticating to the Alpaca Markets API is actually pretty simple - especially when you're using the dev environment we set up previously. There's two components we're going to work through - getting your keys then integrating them into your trading bot.
First, we need to create our
API Key and
Secret Key in Alpaca Markets. This allows us to use their key-based authentication to access their data (and in future episodes our trading history). To do this:
- Log in to your Alpaca account and choose the "Paper" option
- In the Paper section, find the button "View API Keys" and click it
- Generate your keys
Alpaca Markets Authentication: Add the Keys to Your Trading Bot
Next, we're going to add these keys to your GitHub Codespace using the GitHub Encrypted Secrets. To do this:
- Navigate to your GitHub
- Click on your profile picture, then settings
- Navigate down to the Codespaces section, then click on "New secret"
- Add your API Key, calling it "ALPACA_API" (CAPS included)
- Add your API Secret Key, calling it "ALPACA_SECRET_API" (caps included)
At this point, if your Codespace is open, you'll get a pop up notifying you that a new secret has been added, and that to use it you need to reload the environment.
Once both secrets are added, reload the environment.
Update Your Trading Bot Codespace
Let's update your Codespace so that it's ready to develop your trade ideas.
We'll start by adding a virtual environment or venv to your code. This will allow you to lock your environment so that it works as expected.
To do this in your Codespace:
- Navigate to the top menu
- Select 'Show and Run Commands'
- Search for 'Python: Create Environment'
- Create the environment
Next, update your requirements.txt to include the following:
The Python requests library gives you the ability to make HTTP requests to HTTP APIs like Alpaca.
Now, run the command
pip install -r requirements.txt in your terminal to install the library.
Update Your Trading Bot Code
Let's connect your trading bot to Alpaca Markets so you can begin bringing your trading ideas to life!
First, we'll add in the environment variables to make your Alpaca API Key and API Secret Key available. Do this by adding the following to your
Next, we'll add a Python function to the code that:
- Receives a url that relates to an Alpaca Markets HTTP endpoint
- Adds your authentication
- Retrieves the data
- Handles any errors
- Converts the response to json
- Returns the result
Here's the code:
In a little bit, we'll be using this code to get the information we need for your algorithmic trading bots.
Get Historic Data for Your Stock Trading Bots
In order to retrieve historical pricing data for your trading bots, we need to update your Codespace a little further. This will allow us to view the data in a much easier format, which should allow you to make more effective and profitable trades.
Python Pandas: Bringing Trading Strategies to Life Since 2008
To do this, we'll be leveraging a famous Python library called Pandas. Pandas is one of the most popular and widely used tools for data analysis in Python, with over 20 million downloads per month. It was created by a person called Wes McKinney in 2008 while he was working at a quantitative investment firm. He needed a high performance, flexible tool to perform analysis on financial data, and Panda's was the result.
Panda's is truly unique in the data analysis world. It is extremely flexible, fast, and powerful. Its wide usage means that there are integrations with pretty much any trading platform you can imagine.
How do I add Python Pandas to my Algorithmic Trading Bot?
Start by updating your requirements.txt to include pandas. Here's what it should look like:
Now, run the same command in your terminal to install it in your Codespace virtual environment:
pip install -r requirements.txt
How Do I Retrieve Historical Market Data from Alpaca Markets?
We'll be retrieving Klines, or candlesticks, from Alpaca Markets for our trading bots stocks.
Start by adding the following line to line three of your
Now add the following function to the bottom of your
This is a pretty powerful function, and there's a lot going on, so I'll quickly outline it for you.
- Receives a list of the stocks and timeframe you wish to retrieve, along with information about the number of candlesticks you want to retrieve, and the start and end date of your function.
- Checks your inputs to make sure they are in the right format, and no silly errors like the start_date being ahead of the end_date have occurred.
- Converts your python list object into a comma separated string (required for Alpaca API)
- Sets your start and end date to the right format
- Converts all of your data into a dictionary
- Queries the API
- Converts the data into a pandas DataFrame and then updates the fields so that they're nice and easy to read for you and for your algo trading bots.
- Returns the DataFrame
Retrieve Data from the Financial Markets
So far, you've done a lot of work setting up your algo trading bot to automatically retrieve market data for your trading algorithms. Now we're going to see it in action!
Set Your Trading Bot to Auto Run
When we set up the dev environment in a previous episode, I asked you to create a file called app.py. So far it's been hanging out doing absolutely nothing. Let's change that.
app.py and delete the code
Now, add the following code:
Here's what happened when I ran this code:
Scaling Up Your Automated Trading Systems
With this code in hand, feel free to have a little play with the input parameters, located on lines 5-7.
How to Retrieve Market Data for Multiple Stocks
For instance, many of the best stock trading bots use multiple stocks. To do this, you can alter line 5 so that it includes any active stocks you want to try. For instance, you could get the original FAANG by altering this line to:
symbols = ["AAPL", "GOOGL", "META", "NFLX", "AMZN"]
How to Retrieve Price Data for Different Timeframes
Alternatively, many experienced traders like trade different, smaller timeframes. To do this, you could alter line 7 to retrieve the 30-minute timeframe by change it to this:
timeframe = "30min"
How to Retrieve More Candlesticks for Technical Analysis
If you're into technical analysis, you might want to expand the number of candlesticks you want to retrieve. This will allow your investment strategies to consider a greater range of variables and is also quite useful for machine learning algorithms.
To do this, you could change line 6 to retrieve more candles, for instance:
max_number_of_candles = 1000
Why not all?
Ultimately, most successful traders I've observed use a combination of the variables I've mentioned above to predict future trends.
What's the Next Step for Your Algo Trading Bot?
With the conclusion of this episode, you've got everything you need to take your stock trading bot software to the next level.
Check out my blog to see how you can:
- Add a variety of technical indicators to your trading bot
- Use data to make informed trading decisions and identify trading opportunities
- Take your analysis and turn it into a trading strategy
- Create your own AI stock trading bot
- Integrate automated trading so that you can capitalize on market trends
- Use TradeOxy to take everything we've learned and turbocharge it
About Me and About Us
TradeOxy is a trading assistance website I've cofounded with Mitsi, and we're on a mission to democratize algorithmic trading. Sign up today if that mission resonates with you!