Scott Howard Consulting is now Moss Rock Solutions! Click here to learn more!
SHC Logo
Schedule a call

INTRO TO CURL + FILEMAKER

good curl logo

Background

Alex is a film enthusiast and hobbyist. He takes pride in the fact that he’s seen over 500 movies to-date. There’s only one problem: he’s also a full-time accountant. This means Alex doesn’t have time like he used to in his college days to watch movies as he would like. With this constraint in mind, Alex has sought to find some quick, yet organized way to track every movie he’s ever seen. He’s tried a plethora of note-taking apps and websites, but each has its own set of limitations and drawbacks.

Fortunately for Alex, he’s recently stumbled across FileMaker. In his free time, he’s been building his own personal database of films both seen and unseen (yet want to see). Since his free time is limited, he’s brought you, an experienced and exceptionally charming FileMaker developer, along.

The End Goal

Alex wants to add two features to his personal database of films: metadata about each film (i.e. Plot, Genre, Director, Actors, etc.) and its respective poster/image. Since Alex also wants to someday make blog posts about the films he watches, he wants to have access to the actual image file for each film’s poster. He explains that although time-consuming, he can search the web for the plot of each film and download it’s poster that way. But there’s got to be a better way, right? Of course. You’re a FileMaker developer after all.

This article assumes that you are familiar enough with FileMaker Pro that you can create and edit scripts and work with calculations. It will help if you know a little about JSON and how to work with the JSON functions built into FileMaker.

Step 1: Identify a data source

After some research, you stumble across the OMDB (Open Movie Database) API. Using this free API, you’ll find that you can gain access to both a film’s metadata and its respective image/posters. Go ahead and register for the API by clicking the API Key tab and filling out the form to register for the API. You’ll be sent an email with an activation link and an API Key. As far as the API Key– keep it secret, keep it safe.

With that being said, let’s dive right in. This is cool and all, but how do we pull this data into FileMaker? Enter cURL.

What is cURL?

In short, cURL is a free and open-source internet transfer engine. It’s used on tens of billions of devices worldwide and includes a robust set of features. Best of all, it’s supported natively in FileMaker with no need for additional plugins or add-ons.

While cURL is a robust engine, it can essentially be broken down into three key components:

  • url: the URL to transfer data to/from
  • options: request, headers, data, etc.
    • request: request types such as GET, POST, DELETE, etc.
    • headers (optional): information, such as MIME types, about the data being transferred
    • data (optional): data to transfer
    • etc…
  • response: the results of the cURL request

Though this may look complicated, FileMaker makes cURL requests super easy and highly customizable. For a comprehensive list of features, check FileMaker’s supported cURL options.

Step 2: Map data to database

Using cURL, we will need to map data from the OMDB API mentioned previously into our FileMaker database. A good place to start is to check an example API response on the OMDB API’s website. We can expect the response to read something like this:

{
  "Title":"The Dark Knight",
  "Year":"2008",
  "Rated":"PG-13",
  "Released":"18 Jul 2008",
  "Runtime":"152 min",
  "Genre":"Action, Crime, Drama",
  "Director":"Christopher Nolan",
  "Writer":"Jonathan Nolan, Christopher Nolan, David S. Goyer",
  "Actors":"Christian Bale, Heath Ledger, Aaron Eckhart",
  "Plot":"When the menace known as the Joker wreaks havoc and chaos on the people of Gotham, Batman must accept one of the greatest psychological and physical tests of his ability to fight injustice.",
  "Language":"English, Mandarin",
  "Country":"United States, United Kingdom",
  "Awards":"Won 2 Oscars. 159 wins & 163 nominations total",
  "Poster":"https://m.media-amazon.com/images/M/MV5BMTMxNTMwODM0NF5BMl5BanBnXkFtZTcwODAyMTk2Mw@@._V1_SX300.jpg",
  "Ratings":[
    {"Source":"Internet Movie Database","Value":"9.0/10"},
    {"Source":"Rotten Tomatoes","Value":"94%"},
    {"Source":"Metacritic","Value":"84/100"}
  ],
  "Metascore":"84",
  "imdbRating":"9.0",
  "imdbVotes":"2,403,192",
  "imdbID":"tt0468569",
  "Type":"movie",
  "DVD":"14 Jun 2010",
  "BoxOffice":"$534,858,444",
  "Production":"Syncopy",
  "Website":"N/A",
  "Response":"True"
}

Thankfully, this API’s response includes both the necessary metadata (Plot, Genre, Director, Actors, etc.) and a Poster value. This means we’ll map the metadata value to the Films::Plot, Films::Genre, Films::Director, and Films::Actors fields and the Poster to our Films::Poster container field in our database. Having identified this, we’re ready to get started with our first cURL request.

Step 3: Create a cURL Request and Store Results

Words to know:

  • Header: Information about the data being requested or transferred.
  • Endpoint: A specific path on an API, usually dedicated to one specific feature or function.

The best place to start when creating a cURL request is by opening FileMaker’s Script Workspace and creating a new script. We’ll begin by getting a film’s metadata.

  1. Create a new Script “API > Get Film Metadata”.

  2. In the script, enable Set Error Capture to trap for errors and specify the URL to GET. Be sure to replace {{YOUR_API_KEY}} with your unique API key (see Step 1: Identify a Data Source for more information)

1.  # Perform a simple GET Request
2.  Set Error Capture [ On ]

3.  # Specify the API Key
4.  Set Variable [ $API_KEY ; Value: "{{YOUR_API_KEY}}" ]

5.  # Specify the URL to GET
6.  Set Variable [ $URL ; Value: "http://www.omdbapi.com/?apikey=" &
    $API_KEY & "&t=" & Films::Title & "&y=" & Films::Release_Year ]

Notice the t= and y= parameters in the URL, as they are necessary parameters in this case. These are defined in the API as the Movie Title and Year of Release, respectively. Luckily, Alex has already been keeping track of each Film’s title and year of release.

  1. Specify cURL Options and Headers.
...
7.  # Specify cURL Options and Headers
8.  Set Variable [ $dump_headers ; Value: "" ]
9.  Set Variable [ $curl_options ; Value: "-X GET -D $dump_headers --show-error" ]

A few things to notice:

  • $dump_headers: this will contain any HTTP headers that cURL provides to us. Not every API returns a valid response, so it’s best practice to capture headers this way, as it provides a fail safe in case things go awry.
  • $curl_options: this is where the bulk of our cURL request will go. Remember how we discussed the different cURL components earlier? This is where (almost) all those components are set.
    • -X GET is where we specify the request type
    • -D $dump_headers is where we tell cURL “hey you, when you get any http headers, put them here”.
      --show-error is where we tell cURL that we care about errors and want to handle things ourselves
  1. Now for the fun part: perform the cURL request and get some data back.
...
10.  # Init the response
11.  Set Variable [ $response ; "{}" ]
12.  # Perform the request and get the results
13. Insert from URL [ Select ; With dialog: Off ; Target: $response ; $URL ; cURL options: $curl_options ]

14. # Capture any errors
15. Set Variable [ $errors ; Value: Get( LastExternalErrorDetail ) ]

Here, we are reaching out to the API endpoint defined in $URL, giving it the instructions we specified in $curl_options, and storing the results in $response. Since any potential errors will occur outside of FileMaker, you’ll want to use FileMaker’s Get( LastExternalErrorDetail ) to get details about potential errors with our cURL request. It’s best to capture these in a variable and store them somewhere in the database (in order to preserve the error message).

  1. Store the results in the database
...
16. # Set the metadata in the Films::Metadata field
17. Set Field [ Films::Metadata ; Value: JSONFormatElements ( $response ) ]

18. #  From the metadata, strip out the Plot, Director, ID, Actors, Genre, and the Poster URL
19. Set Field [ Films::Plot ; Value: JSONGetElement ( Films::Metadata ; "Plot" ) ]
20. Set Field [ Films::Director ; Value: JSONGetElement ( Films::Metadata ; "Director" ) ]
21. Set Field [ Films::Actors ; Value: JSONGetElement ( Films::Metadata ; "Actors" ) ]
22. Set Field [ Films::Genre ; Value: JSONGetElement ( Films::Metadata ; "Genre" ) ]
23. Set Field [ Films::Poster_URL ; Value: JSONGetElement ( Films::Metadata ; "Poster" ) ]
24.  Set Field [ Films::imdb_id ; Value: JSONGetElement ( Films::Metadata ; "imdbID" ) ]

25. # Track any errors (if applicable)
26. Set Field [ Films::API_Errors ; Value: $errors ]
27. Commit Records/Requests [ With dialog: Off ]
28. Exit Script [ Text Result: True ]

Here, we are simply storing the results of our cURL request (and any errors) in the database. If any of the JSONFormatElements or JSONGetElement functions look unfamiliar, check FileMaker’s working with the JSON functions guide. Though not required, when working with data external to FileMaker it is recommended to keep track of any sort of IDs provided. In this case, our cURL request returns an imdbID value, so we should keep track of that in our database.

One more thing– notice how we are storing the Poster value in the Films::Poster_URL field and not the Films::Poster container field. This is important. If you look in the cURL response, the Poster is actually a URL, not an image file. Fear not, we’ll handle this later.

  1. Your finished script should look like this:
1.  # Perform a simple GET Request
2.  Set Error Capture [ On ]

3.  # Specify the API Key
4.  Set Variable [ $API_KEY ; Value: "{{YOUR_API_KEY}}" ]

5.  # Specify the URL to GET
6.  Set Variable [ $URL ; Value: "http://www.omdbapi.com/?apikey=" &
    $API_KEY & "&t=" & Films::Title & "&y=" & Films::Release_Year ]

7.  # Specify cURL Options and Headers
8.  Set Variable [ $dump_headers ; Value: "" ]
9.  Set Variable [ $curl_options ; Value: "-X GET -D $dump_headers --show-error" ]

10.  # Init the response
11.  Set Variable [ $response ; "{}" ]
12.  # Perform the request and get the results
13. Insert from URL [ Select ; With dialog: Off ; Target: $response ; $URL ; cURL options: $curl_options ]

14. # Capture any errors
15. Set Variable [ $errors ; Value: Get( LastExternalErrorDetail ) ]

16. # Set the metadata in the Films::Metadata field
17. Set Field [ Films::Metadata ; Value: JSONFormatElements ( $response ) ]

18. #  From the metadata, strip out the Plot, Director, ID, Actors, Genre, and the Poster URL
19. Set Field [ Films::Plot ; Value: JSONGetElement ( Films::Metadata ; "Plot" ) ]
20. Set Field [ Films::Director ; Value: JSONGetElement ( Films::Metadata ; "Director" ) ]
21. Set Field [ Films::Actors ; Value: JSONGetElement ( Films::Metadata ; "Actors" ) ]
22. Set Field [ Films::Genre ; Value: JSONGetElement ( Films::Metadata ; "Genre" ) ]
23. Set Field [ Films::Poster_URL ; Value: JSONGetElement ( Films::Metadata ; "Poster" ) ]
24.  Set Field [ Films::imdb_id ; Value: JSONGetElement ( Films::Metadata ; "imdbID" ) ]

25. # Track any errors (if applicable)
26. Set Field [ Films::API_Errors ; Value: $errors ]
27. Commit Records/Requests [ With dialog: Off ]
28. Exit Script [ Text Result: True ]

Congrats! You should now have all the film’s metadata stored and mapped out to the proper fields.

Task: Get Film Metadata (Complete) ✅

Wow! Look at you. You just went 0-60 with cURL. Now there’s only one problem: the poster is still missing! Don’t worry, you’re now a cURL protogé. You got this.

Step 4: Download the Poster File with cURL

To download the poster file, you’re going to need two things– a url pointing to the file to download and a valid container field. Oddly enough, you already have both. Remember how I said we need to store the Poster value in the cURL response in the Films::Poster_URL field? Well now you’re one step ahead.

Again, open the Script Workspace and do the following:

  1. Create a new script called “API > Download Poster File“

  2. In the script, enable Set Error Capture to trap for errors and specify the URL to GET. In this case, the URL we want is the Poster URL stored in the Films::Poster_URL field.

1.  # Perform a simple GET Request
2.  Set Error Capture [ On ]

3.  # Specify the URL to GET
4.  Set Variable [ $URL ; Value: Films::Poster_URL ]
  1. Specify cURL Options and Headers.
...
5.  # Specify cURL Options and Headers
6.  Set Variable [ $dump_headers ; Value: "" ]
7.  Set Variable [ $curl_options ; Value: "-X GET -0 -L -D $dump_headers --show-error" ]

This should look somewhat familiar. However, there are two new flags being set here: -0 and -L.

  • -0: this is the “binary” flag in cURL. This tells cURL, “I want to download the file exactly as-is, nothing fancy. Download using good ole 1s and 0s”.
  • -L: this is the “redirect” flag in cURL. This tells cURL to follow any redirects it may encounter. While this is not always necessary, it is a good idea to include.
  1. Download the file directly to the container field
...
8.  # Perform the request and download the file
9.  Insert from URL [ Select ; With dialog: Off ; Target: Films::Poster ; $URL ; cURL options: $curl_options ]

10. # Capture any errors
11. Set Variable [ $errors ; Value: Get( LastExternalErrorDetail ) ]

Notice here how we set the results of the cURL request directly into the Films::Poster field. Since we are downloading a file directly using binary, we can send our results straight to a container field.

  1. Store any errors
...
12. # Track any errors (if applicable)
13. Set Field [ Films::API_Errors ; Value: $errors ]
14. Commit Records/Requests [ With dialog: Off ]
15. Exit Script [ Text Result: True ]
  1. Your finished script should look like this:
1.  # Perform a simple GET Request
2.  Set Error Capture [ On ]

3.  # Specify the URL to GET
4.  Set Variable [ $URL ; Value: Films::Poster_URL ]

5.  # Specify cURL Options and Headers
6.  Set Variable [ $dump_headers ; Value: "" ]
7.  Set Variable [ $curl_options ; Value: "-X GET -0 -L -D $dump_headers --show-error" ]

8.  # Perform the request and download the file
9.  Insert from URL [ Select ; With dialog: Off ; Target: Films::Poster ; $URL ; cURL options: $curl_options ]

10. # Capture any errors
11. Set Variable [ $errors ; Value: Get( LastExternalErrorDetail ) ]

12. # Track any errors (if applicable)
13. Set Field [ Films::API_Errors ; Value: $errors ]
14. Commit Records/Requests [ With dialog: Off ]
15. Exit Script [ Text Result: True ]

Congrats! You should now have the film’s official poster stored in the container field.

Task: Download Poster File (Complete) ✅

And that’s it! It might be good to add some buttons to the interface, just so Alex has an easy way of running these processes himself. You’re a rockstar and a cURL-fluent developer.

Conclusion

Hopefully by now you can see just how powerful cURL really is. I mean, after all you were able to download a file in just 15 script steps. This could be further simplified and even done in only 1 script step. Hint– it would look something like this:

1. Insert From URL [ Select ; With dialog: Off ; Target: {{myTable::MyField}} ; {{URL}} ; cURL options: {{options}} ]

Hopefully, this has piqued your interest in cURL. If you want to learn more, check out some of the excellent resources below. Farewell and happy FileMaking!

Resources:

Downloads