Rails Tricks
Archive
22 Aug
Active Record upsert - Tricks Issue 16
This week, I will show you how to make data imports blazing fast with Active Record.
I recently built a subscriber import functionality into the newsletter tool I am working on. The requirements were the following:
- a CSV file is uploaded
- then on the next screen the columns of the file can be mapped to the importable attributes
- a subscriber is created unless it already exists on the list
This second part could have a few naive implementation with a query to check if a matching record already exists, or by doing a find_or_create_by with each row of the import, but luckily Active Record supports upsert and even have an upsert_all method.
What upsert does in SQL, is it either updates a record if found or creates a new one, and since it happens on the database level, it is way more performant than doing it in Ruby.
The Active Record method takes the attributes as the first parameter and a list of optional parameters:
- on_duplicate: a SQL update sentence that will be used on conflict. By default it is update
- update_only: a list of column names to update on conflict. nil by default which means it updates all columns provided in the query.
- returning: an array of attributes to return for all successfully returned records. By default it returns the primary key. This works in PostgreSQL only.
- unique_by: by default, rows are considered to be unique based on every unique index on the table, but in PostgreSQL and SQLite, you can speficy the attributes or an index name you want to use. If you provide the attribute or attributes, you have to have a unique index on them.
- record_timestamps: whether to record the timestamps on the table or not. This uses the model’s record_timestamps by default.
Now let’s see an actual example. Let’s imagine we are receiving a list of subscribers in the params and we want to upsert them:
The above code will be pretty performant.
That’s it for this week!
15 Aug
Rails static pages - Rails Tricks Issue 15
Hi there,
This week I want to show you how I add static pages in a Rails application, since I just did this recently.
I like to put all of them into the same controller so they are nicely separated, so I generate a new controller called StaticPagesController. Then I add my routes:
As I found out, this feature called “implicit rendering”. What happens is, when Action Controller is processing a request, it is trying to find the appropriate method for the action it receives from the router:
11 Jul
Modifying Action Text markup - Rails Tricks Issue 14
Hi there,
I am working on a newsletter tool(Pombo) and this week, I want to share how I solved a problem I came across last week while working on it.
The problem: the newsletter tool needs to support importing past issues from other tools. These imported issues might contain references to images hosted at the other provider, and during the import, I need to move those images into Pombo because once the account on the old service is deleted, the images won’t be accessible anymore.
In Pombo, I use Action Text to store the body of a newsletter issue, and I use Active Storage for the images. My initial plan was to parse the HTML output of the Action Text field with Nokogiri, extract the image tags, create an Active Storage blob from them, and replace the image tag’s src in the markup to the blob’s URL.
Then I decided to look at Action Text’s source code to see how it works under the hood, and it turned out it will be way easier to do what I need than I expected.
An Action Text field has a “body” attribute, which is an instance of ActionText::Content which has a fragment method, which returns an instance of ActionText::Fragment and that’s a wrapper around the Nokogiri parsed markup.
Now putting this all together, to find all image tags on a content attribute backed by Action Text, we can just do the following:
Digging a bit more into Action Text, I figured out I need to create an ActionText::Attachment from the blob and replace the nokogiri node with that: And finally, we need to save the changes on the content. Here is the full snippet: Based on what I learned from figuring out this solution, it will be pretty easy to add an outgoing link validator to make sure a newsletter doesn’t contain broken links.
I hope you enjoyed this, until next time!
04 Jul
Automate some of your security - Rails Tricks Issue 13
This week, I will show you how you can automate some of the security necessities of a Rails application. If you follow this guide, you will be safe from one of the OWASP Top 10 security issues(A9-Using Components with Known Vulnerabilities) and lower the chances of having other vulnerabilities in your codebase. Let’s get into it.
Using Components with Known Vulnerabilities is in the OWASP Top 10, but automating a notification about gems with known vulnerabilities is very easy. The bundler-audit gem covers us there with the bundle-audit command:
[~/] bundle-audit --help Commands: bundler-audit check [DIR] # Checks the Gemfile.lock for insecure dependencies bundler-audit download # Downloads ruby-advisory-db bundler-audit help [COMMAND] # Describe available commands or one specific command bundler-audit stats # Prints ruby-advisory-db stats bundler-audit update # Updates the ruby-advisory-db bundler-audit version # Prints the bundler-audit version
If you put a check for bundle audit --update to your CI workflow, it will check your app for vulnerable dependencies and your pipeline will fail. Additionally, if you use yarn to manage your javascript dependencies, you can use yarn audit to check your dependencies for any known vulnerability.
Here is an example GitHub Action file to do this:
# .github/workflows/bundle-audit.yml name: Bundle Audit on: pull_request: schedule: - cron: "0 0 * * *" jobs: base: runs-on: ubuntu-latest strategy: fail-fast: false steps: - name: Checkout code uses: actions/checkout@v2 - name: Setup Ruby uses: ruby/setup-ruby@v1 with: ruby-version: 3.2 bundler-cache: true - name: Install bundler-audit run: gem install bundler-audit - name: Check dependencies with known vulnerabilities run: bundle-audit --update - name: Check javascript dependencies run: yarn audit
The above action runs bundle audit and yarn audit on every pull request and at midnight every day. You might need to adjust the Ruby version above to the one you are on.
Another low-hanging fruit to improve the security posture of a Ruby on Rails application is to set up static code analyses for potential security issues. There are two gems to help with this: brakeman and spektr(DISCLAIMER: I am the author of this gem). These gems analyze your code for potentially vulnerable code and can help to find SQL injections, XSS, and quite a few other issues.
Using on CI brakeman is more ideal, because it supports ignoring false positives out of the box. Spektr is targeted more towards security professionals running it on a codebase during an assessment.
Here is an example GitHub Actions file to run brakeman on your codebase on every pull request and once every day:
# .github/workflows/brakeman-scan.yml name: Brakeman Scan on: - pull_request: - schedule: - cron: "0 0 * * *" jobs: base: runs-on: ubuntu-latest strategy: fail-fast: false steps: - name: Checkout code uses: actions/checkout@v2 - name: Setup Ruby uses: ruby/setup-ruby@v1 with: ruby-version: 3.2 bundler-cache: true - name: Install brakeman run: gem install brakeman - name: Static code analyses for security run: brakeman
That’s it for this week.
27 Jun
Customizing the Rails console
Hi there,
I started to deploy my latest pet project with MRSK and I realized that I miss the customizations from my Rails console on the server. And I thought it might be useful to share how I customize my Rails console, so here we go.
The default Rails console is built on top of IRB, so to customize it, we need to look at IRBs documentation. The docs say that we can either create a global .irbrc in the home directory of the user, or have a project specific one in the project directory, but the home directory one will have a presedence. In my example, I keep it in the project, so I can copy it with docker to the containers when I deploy.
This .irbrc file is a evaluated as ruby, so to do our customizations, we can use our favorite language. The first line in my config is to disable autocompletion because I dislike it:
IRB.conf[:USE_AUTOCOMPLETE] = false
For my dockerized project, I also disable IRB history, since the containers are short-lived, and the history won’t be kept long and it also prevents the exception on the console exit when it fails to write to the file.
IRB.conf[:SAVE_HISTORY] = false if Rails.env.production?
The next thing I have in my file is a prompt customization. I use the pastel gem to colorize the prompt based on the environment:
require "pastel" pastel = Pastel.new prompt = case Rails.env when "development" pastel.black.on_green.bold.dim Rails.env when "production" pastel.white.on_red.bold.dim Rails.env end # defining custom prompt IRB.conf[:PROMPT][:CUSTOM] = { # name of prompt mode :PROMPT_I => "#{prompt}>> ", # simple prompt :PROMPT_S => "#{prompt}* ", # prompt for continuated strings :PROMPT_C => "#{prompt}? ", # prompt for continuated statement :RETURN => " => %s\n" # format to return value } IRB.conf[:PROMPT_MODE] = :CUSTOM
As you can see, I don’t do anything fancy in my prompt, just outputting the Rails environment on a green or red background based on the environment. You can get it fancier by using the following special characters in the prompt:
%N # command name which is running %m # to_s of main object (self) %M # inspect of main object (self) %l # type of string(", ', /, ]), `]' is inner %w[...] %NNi # indent level. NN is digits and means as same as printf("%NNd"). # It can be omitted %NNn # line number. %% # %
That’s it for this week.