Software Development – Page 11

With as rough as the Fitbit API is, it's helpful to have a guide on how to navigate the authentication process. As result, I figured I'd make one! The Fitbit API Ecosystem Sucks I've been using a Fitbit for nearly 7 years, and I've never been interested in getting involved in development. That said, I recently decided to try programmatically pulling data using the Fitbit API and found it to be a nightmare. There are a couple reasons for this. First, there is only one Python library built to work with the Fitbit API, and it's largely inactive. At the time of writing, the last commit was nearly three years ago (August 12, 2019). Since then, the Fitbit API has expanded and changed—rendering portions of the Python library useless. To make matters worse, the Fitbit development documentation is pretty rough. As a novice to OAuth authentication, I was very confused how to even get started. Once I managed to generate the access tokens, I ran into some other issues which I'll cover some other time. Today, however, I just wanted to cover that first step: how to generate the Fitbit access tokens. I'll show you two ways to do it: one manual and one automated. Let's get into it! OAuth 2.0: Token Generation If you ever plan to pull data from the Fitbit API, you're going to need to do two things. First, you'll need to create an app (and a Fitbit account, if you don't already have one). I won't cover that here, but the process is described nicely here. Then, you'll need to generate access tokens for when you request data. This article will cover the token portion. Manually Creating Access Tokens The one thing I'll give Fitbit credit for is their OAuth 2.0 tutorial page. It's basically designed to walk you through the process of creating access tokens for your app. Unfortunately, I cannot share the link for this tutorial with you because it includes my tokens as parameters. That said, here's an empty example page. On this page, you'll see some boxes with information that you'll need to provide: Here, you can insert the client ID, client secret, and redirect URI of your app—all of which can be found on your app page. Alternatively, you can click the "OAuth 2.0 tutorial page" link at the bottom of your app page. From here, Fitbit will generate a link you can use to get a code that we'll reuse later. Here's the base of the link. As with the tutorial page link, this one will include a bunch of parameters such as the client ID, redirect URI, and scope. If all is well, you can click that link to retrieve your code. For example, if you set your redirect URI to 127.0.0.1:8080, then that link should redirect you to a link that looks like this: http://127.0.0.1:8080/?code=e62088d9c45d15212ded905c79f8b835e35dca36#=. Basically, you just want to copy the code out of the link (i.e., "e62088d9c45d15212ded905c79f8b835e35dca36"). With the code copied, you can head back to the tutorial page and drop it into the "code" box (e.g., step 1A). From here, Fitbit will generate you a curl command. This command is meant for a unix/linux based system, so you may have trouble running it on windows. I was able to drop it into git bash on my Windows 11 machine. There are probably other ways of getting it done. Regardless, as long as you're able to run the curl command, you'll get a JSON response. I don't want to share exactly what that looks like because I'd be sharing keys, but here's an example (note: I mangled or outright replaced all codes below): $ curl -i -X POST \ > -H 'Authorization: Basic MjM4C1BZOjQwN2M4fjg0MzViNDMzOTdhZ7ZlZjk3ZTwwMmQ9NzBm' \ > --data "clientId=2419B1" \ > --data "grant_type=authorization_code" \ > --data "redirect_uri=http%3A%2F%2F127.0.0.1%3A8080%2F" \ > --data "code=e62088d9c45d15212ded905c79f8b835e35dca36" \ > -H 'Content-Type: application/x-www-form-urlencoded' \ > https://api.fitbit.com/oauth2/token % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 724 0 587 100 137 2266 528 --:--:-- --:--:-- --:--:-- 2931HTTP/2 200 date: Sun, 07 Aug 2022 23:55:04 GMT content-type: application/json;charset=UTF-8 vary: Origin,Accept-Encoding cache-control: no-cache, private set-cookie: fct=e72985651bfa435984139609dc6ef05b; Path=/; Secure; HttpOnly set-cookie: JSESSIONID=87AE3F3BED5B62450C57CABF1DC3EB3E.fitbit1; Path=/; Secure; HttpOnly content-language: en-US x-frame-options: SAMEORIGIN via: 1.1 google cf-cache-status: DYNAMIC expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" server: cloudflare cf-ray: 7373f0e82c358cb3-EWR {"access_token":"some_access_code","expires_in":28800,"refresh_token":"some_refresh_token","scope":"activity respiratory_rate sleep oxygen_saturation settings weight nutrition temperature social location heartrate profile","token_type":"Bearer","user_id":"some_user_id"} You can then dump the JSON directly into the parse box on the OAuth tutorial page. Life is good! You have your access token and refresh token. Automatically Creating Access Tokens I found the process above extremely painful to repeat more than once, and it became a regular occurrence for me as tokens would seemingly stop working at random. As a result, I decided I wanted to automate the token creation process using Python. Fair warning: I'm not sure the way I chose to automate token creation is practical or even encouraged. In fact, it seems like there's an implicit grant option that makes requesting codes a bit easier (albeit less secure). That said, I'll share what I did anyway. Retrieving the Code Right off the bat, we're going to way to automate the "code" creation process. To do that, I made use of Selenium to actually launch a browser with the authorization URL. Here's what that looks like: def automate_code_retrieval() -> str: """ Grabs the initial code from the Fitbit website containing the correct scopes. :return: the code as a string """ url = "https://www.fitbit.com/oauth2/authorize" \ "?response_type=code" \ f"&client_id={os.environ.get('FITBIT_CLIENT_ID')}" \ "&redirect_uri=http%3A%2F%2F127.0.0.1%3A8080%2F" \ "&scope=activity%20heartrate%20location%20nutrition%20profile%20settings%20sleep%20social%20weight%20oxygen_saturation%20respiratory_rate%20temperature" \ "&expires_in=604800" # Get URL driver = webdriver.Chrome(service=Service(ChromeDriverManager().install())) driver.get(url) # Complete login form WebDriverWait(driver, 10).until( EC.presence_of_element_located((By.XPATH, "//input[@type='email']"))) username_input = driver.find_element(By.XPATH, "//input[@type='email']") password_input = driver.find_element(By.XPATH, "//input[@type='password']") submit = driver.find_element( By.XPATH, "//form[@id='loginForm']/div/button") username_input.send_keys(os.environ.get("FITBIT_USERNAME")) password_input.send_keys(os.environ.get("FITBIT_PASSWORD")) submit.click() # Get code WebDriverWait(driver, 10).until(EC.url_contains("127.0.0.1:8080")) code = driver.current_url.split("code=")[-1].split("#")[0] driver.quit() return code There are a couple of things I should note here. First, there are a few dependencies we need to install and subsequently import: import dotenv from selenium import webdriver from selenium.webdriver.chrome.service import Service from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.support.ui import WebDriverWait from webdriver_manager.chrome import ChromeDriverManager Here, dotenv allows us to load in some credentials that are saved locally in a .env file. Use load_dotenv() of the dotenv package before the automate_code_retrieval() function is executed to ensure the Client ID and Fitbit username and password are loaded into the environment. Meanwhile, webdriver_manager exists to automatically install the Chrome driver if you don't already have it. Likewise, selenium provides the actual functionality for automatically clicks and whatnot on webpages. With that out of the way, here's how the automate_code_retrieval() function works. It first supplies the URL from the OAuth tutorial page to the browser. It then waits for the page to load before logging a Fitbit user in. The subsequent page that loads is the redirect URI (i.e., 127.0.0.1:8080) with the code attached. We parse that URL and return the code directly. Retrieving the Tokens With our code properly retrieved, we can officially move on to getting our access tokens. That process is thankfully much easier. All we have to do is make a post request to the token URL using all of the data we've collected up to this point. Here's what that looks like: def automate_token_retrieval(code: str): """ Using the code from the Fitbit website, retrieves the correct set of tokens. """ data = { "clientId": os.environ.get("FITBIT_CLIENT_ID"), "grant_type": "authorization_code", "redirect_uri": "http://127.0.0.1:8080/", "code": code } basic_token = base64.b64encode( f"{os.environ.get('FITBIT_CLIENT_ID')}:{os.environ.get('FITBIT_CLIENT_SECRET')}".encode("utf-8") ).decode("utf-8") headers = { "Content-Type": "application/x-www-form-urlencoded", "Authorization": f"Basic {basic_token}" } response = requests.post(data=data, headers=headers, url="https://api.fitbit.com/oauth2/token") keys = response.json() dotenv.set_key(".env", "FITBIT_ACCESS_TOKEN", keys["access_token"]) dotenv.set_key(".env", "FITBIT_REFRESH_TOKEN", keys["refresh_token"]) dotenv.set_key(".env", "FITBIT_EXPIRES_AT", str(keys["expires_in"])) Again, there is a little bit of work that has to occur before we can make this function work. For example, we need to incorporate another library: import requests The requests library allows us to make a post request in the form of the curl command we used earlier. This command is somewhat nontrivial to implement. Basically, we need to pass four lines of data: the client ID, the grant type, the redirect URI, and the code from the previous step. In addition, we need to provided an authenticated header which is comprised of a base 64 mangling of the client ID and the client secret. That said, once the post request is properly formed, a JSON will be returned just like the one from before. I happen to parse the JSON programmatically and store the keys directly in a local .env file, but you can take those tokens wherever you'd like—within reason. Putting It All Together And just to make it abundantly clear, here's what your code might look like: import dotenv import requests from git import Repo from selenium import webdriver from selenium.webdriver.chrome.service import Service from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.support.ui import WebDriverWait from webdriver_manager.chrome import ChromeDriverManager def automate_code_retrieval() -> str: """ Grabs the initial code from the Fitbit website containing the correct scopes. :return: the code as a string """ url = "https://www.fitbit.com/oauth2/authorize" \ "?response_type=code" \ f"&client_id={os.environ.get('FITBIT_CLIENT_ID')}" \ "&redirect_uri=http%3A%2F%2F127.0.0.1%3A8080%2F" \ "&scope=activity%20heartrate%20location%20nutrition%20profile%20settings%20sleep%20social%20weight%20oxygen_saturation%20respiratory_rate%20temperature" \ "&expires_in=604800" # Get URL driver = webdriver.Chrome(service=Service(ChromeDriverManager().install())) driver.get(url) # Complete login form WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//input[@type='email']"))) username_input = driver.find_element(By.XPATH, "//input[@type='email']") password_input = driver.find_element(By.XPATH, "//input[@type='password']") submit = driver.find_element(By.XPATH, "//form[@id='loginForm']/div/button") username_input.send_keys(os.environ.get("FITBIT_USERNAME")) password_input.send_keys(os.environ.get("FITBIT_PASSWORD")) submit.click() # Get code WebDriverWait(driver, 10).until(EC.url_contains("127.0.0.1:8080")) code = driver.current_url.split("code=")[-1].split("#")[0] driver.quit() return code def automate_token_retrieval(code: str): """ Using the code from the Fitbit website, retrieves the correct set of tokens. """ data = { "clientId": os.environ.get("FITBIT_CLIENT_ID"), "grant_type": "authorization_code", "redirect_uri": "http://127.0.0.1:8080/", "code": code } basic_token = base64.b64encode( f"{os.environ.get('FITBIT_CLIENT_ID')}:{os.environ.get('FITBIT_CLIENT_SECRET')}".encode("utf-8") ).decode("utf-8") headers = { "Content-Type": "application/x-www-form-urlencoded", "Authorization": f"Basic {basic_token}" } response = requests.post( data=data, headers=headers, url="https://api.fitbit.com/oauth2/token" ) keys = response.json() dotenv.set_key(".env", "FITBIT_ACCESS_TOKEN", keys["access_token"]) dotenv.set_key(".env", "FITBIT_REFRESH_TOKEN", keys["refresh_token"]) dotenv.set_key(".env", "FITBIT_EXPIRES_AT", str(keys["expires_in"])) if __name__ == "__main__": dotenv.load_dotenv() code = automate_code_retrieval() automate_token_retrieval(code) In addition, you'll want to create a local .env file that looks like this: FITBIT_USERNAME="SOME_FITBIT_USERNAME" FITBIT_PASSWORD="SOME_PASSWORD" FITBIT_CLIENT_ID='SOME_CLIENT_ID' FITBIT_CLIENT_SECRET='SOME_FITBIT_CLIENT_SECRET' FITBIT_ACCESS_TOKEN='SOME_FITBIT_CLIENT_ACCESS_CODE' FITBIT_REFRESH_TOKEN='SOME_REFRESH_TOKEN' FITBIT_EXPIRES_AT='SOME_NUMBER' On the first run, you'll only need the first four values filled out. The script will populate the rest. Happy coding! Know Any Fitbit API Tricks? As I mentioned already, I developed this script to automatically generate the access token and refresh token. For whatever reason, these tokens were expiring on my system, and the refresh token wasn't working. My workaround was to just request a new token if the refresh token ever failed. I realize this is probably against solid principles, but it's a pain otherwise, At any rate, do you have any good tips for working with the Fitbit API. For instance, how are you keeping your keys alive? Do you have to run your program regularly? The access token stops working before I pull data (expires_in is set to 8 hours), which is ever 24 hours. That seems contrary to these notes by Fitbit: If you followed the Authorization Code Flow, you were issued a refresh token. You can use your refresh token to get a new access token in case the one that you currently have has expired. My suspicion is that I'm not properly updating my .env and the current environment during a run, which would cause the script to return an error. That's something I'll have to revisit. This GitHub thread was pretty helpful as well, but I seem to be still having issues. At any rate, thanks for checking this out. Hopefully, this helped some folks—even if you don't use the code I provided. Hopefully, another Fitbit library will come along for Python that is actually usable. In the meantime, this is a nice hack. As always, if you liked this and want to see more like it, consider helping grow the site. Otherwise, take care!