Latest Blog Posts

• Using Embedded Schemas for Easy Peasy Ecto Preloading

  Structs in Elixir are great, they let you define some data structure and lets you do all sorts of nifty stuff like default values. But what if you want to use this particular struct inside of an Ecto query and then preload associations based on a given field? An Example Problem We have a struct called Deal, which is build dynamically from an SQL query. This means that there is no table associated with our Deal structs.

  Dec 28, 2020 Read more...

• Using Common Table Expressions for Temporary Tables in Elixir (Ecto)

  Ocassionally, when you've got lots of business logic defined, you may need to perform some heavy calculations outside of your main SQL query and then join back the calculated result set into your final query to perform some statistical final calculation. Usually, the calculated result set would be in the form of a list of maps or list of tuples. Thankfully, we can use Ecto.Query.with_cte/3 to help with this. With the help of the PostgreSQL function unnest, we can interpolate arrays into the query while also defining the data type for that temporary column.

  Dec 6, 2020 Read more...

• Deploying a Trained SpaCy Model As a Python Sub-Package

  So you've trained your spaCy NLP model, you've got it to the highest accuracy you can push it to, and now you want to use it in your project or product. Now, what do you do? If we were to take a glance at the official spaCy documentation for packaging, they advise using their command line interface to build a deployable Python package containing the model. This results in a package that can then be imported as follows:

  Oct 31, 2020 Read more...

• Translating Data Types Between Elixir and Python (when using Erlport/Export)

  As explained in my original post on connecting Python with Elixir, there are some data translation issues that we will have to deal with when interfacing between the two languages. Thankfully, the Erlport documentation kindly provides a full table of the data type mappings, which we can now use to make our lives easier. Translating On The Python Side When calling Python from Elixir, we will want pass our arguments as so:

  Oct 31, 2020 Read more...

• Using Recursive Functions for Ecto Query Building

  Composability is the name of the game when it comes to writing Ecto queries. With such a beautifully designed DSL given to us, we should make full use of its design to build our queries. A Brief on Recursive Functions Recursive functions are functions that call themselves. There is the risk of functions becoming infinite loops if they are not given exit conditions. Here's an example: def eat(food) when food == "biscuits" do eat(food, "water") end def eat(food) do eat(food, nil) end # return the final digestable result def eat(food, drink) do {food, drink} end In the example, we keep calling the eat function until we end up with a digestable form of input, a food input and drink.

  Sep 16, 2020 Read more...

• Doing Bulk Insertions with Embedded Schemas with Ecto.Repo.insert_all

  You can perform bulk insertions for records that contain embedded schemas by doing the following: alias MyApp.MyEmbed alias MyApp.MyMainSchema # multiple_attrs is a list of map attrs that we want to insert def bulk_create(multiple_attrs) do multiple_attrs = Enum.map(multiple_attrs, fn attrs -> attrs # put timestamps, since they are not auto-generated for us |> Map.put(:inserted_at, NaiveDateTime.utc_now() |> NaiveDateTime.truncate(:second)) |> Map.put(:updated_at, NaiveDateTime.utc_now() |> NaiveDateTime.truncate(:second)) # This will pipe the attributes into the anonymous function # we will do embed-specific operations here |> (fn attrs -> # This creates a struct of MyEmbed on the key :my_embed %{attrs | my_embed: struct(MyEmbed, attrs.

  May 31, 2020 Read more...

• Using Python's Invoke for Simple GitFlow Release Automation

  While developing PillowSkin, I have become used to the habit of creating small releases. This resulted in me always having to repetitively write the commands git checkout master, then git merge develop, then git tag… It does get old after a while when your total release count gets into the hundreds (PillowSkin is at ~180 versions at this point in writing). However, with the power of Python scripting, we can do a little bit of scripting to save us countless cumulative hours of googling for the right commands to type.

  May 18, 2020 Read more...

• Shrink Your Static Website Size by 10x With This Simple Trick (gzip)!

  This “simple trick” is nothing very special: it's our good old friend gzip. However, not all static website hosts automatically compress and serve your website and assets. This may inadvertently result in larger-than-necessry file sizes when creating and publishing your static websites on hosts such as Gitlab Pages, Github Pages, or Netlify. As such, we need to manually compress the files as part of our website build step. In your CI build script, add in the following right after your build script:

  May 17, 2020 Read more...