stream on Poeticoding

ExAws with DigitalOcean Spaces

Wed, 04 Dec 2019 14:39:32 +0000

When you are hosting an Elixir/Phoenix app in DigitalOcean you may want to use Spaces, the DigitalOcean’s cloud storage alternative to AWS S3.

When you are hosting an Elixir/Phoenix app in DigitalOcean you may want to use Spaces, the DigitalOcean’s cloud storage alternative to AWS S3 (If you want to use AWS S3 instead, you can read AWS S3 in Elixir with ExAws).

Elixir Stream and large HTTP responses: processing text

Wed, 03 Apr 2019 16:02:19 +0000

Part 1 - Elixir Stream to process large HTTP responses on the fly
Part 2 - this article

You find the code, of this and the previous article, on poeticoding/httpstream_articles GitHub repo. The code at this repo is not meant for production use, is just part of the examples of these articles.

Elixir Streams to process large HTTP responses on the fly

Wed, 27 Mar 2019 16:48:21 +0000

Part 1 - this article
Part 2 - Elixir Stream and large HTTP responses: processing text

This is the first of a two-part article where we see a handy way of handling async HTTP responses.\

In this first part we see how make an Elixir Stream out of a large HTTP response.
In the second part we will extend the implementation so we can download a large text file, processing lines on the fly.

First and foremost … WHY?

Why should we handle HTTP responses with Streams, when we can just simply use something like HTTPoison.get to receive a response with the data we need?

Processing Large CSV files with Elixir Streams

Thu, 10 Jan 2019 14:21:24 +0000

We need to process a large CSV file of minute by minute volume and prices. Our task is pretty simple: we just want to get the first line of the year 2015, with valid data. At first, this seems an easy task we could tackle with String.split, Enum.map/filter/find. But what happens when the CSV file is large?