Unless you live under a rock, you’ve heard of the key-value store Redis. It’s possible though, you’ve never used it. Maybe you don’t have giant, loaded site needs like Imgur, so you haven’t added it to your stack. That’s cool. In fact, I’m a proponent of not adding an additional cog to a production stack unless it either greatly reduces load on an existing part or makes the code easier to understand. That said, Redis has a reputation for helping out when your site is under giant load. But there’s something I think is great about Redis that makes it useful for small tasks: its simplicity.
Redis has a few different structure types, take Hashes and Lists. For example, using the redis-rb client, you can set values in a Redis hash:
redis.hset "user:1", :name, "Brendon"
redis.hset "user:1", :gender, "male"
redis.hget "user:1", "name" #=> "Brendon"
redis.hgetall "user:1" # => {"gender"=>"male", "name"=>"Brendon"}
redis.hvals "user:1" # => ["male", "Brendon"]
This feels like a simple, hash like primative to work with. Though it is syntatically different than working with a plain old ruby Hash, the feel is similar. I think this is a big win; it’s easy to understand and feels familiar, plus, it’s network distributed.
Check out some List basics:
redis.rpush "comments", "Awesome"
redis.rpush "comments", "Super"
redis.llen "comments" # => 2
redis.lpop "comments" # => "Awesome"
Again, syntatically a little different than using an Array, but still familiar.
Redis itself is very easy to install. You can install it from homebrew, or apt-get or just compile from source. It’s small, installs quickly, and unless you fill it with gigabytes of data, it’s likely you will forget it’s there. I install it by default on most my systems, even little underpowered ones. The reason being, I love Redis for my little one off scripting tasks because of a few features:
- It’s liteweight
- Data has optional expiration so it will clean up after itself for free
- Its commands for get/set, lists, and pub/sub are great for basic IPC.
Here’s some examples. I’m excluding exception handling and Redis watches/multi/transaction concepts to focus on the core idea.
Redis is great at deduping. Say you want to write a script that pushes campfire notifications to gently remind the team to keep commit messages under 50 characters, but not resend reminders in the case of forced pushes, rebases, merges, etc:
# Generate your own digest to avoid git hash changing
id = Digest::MD5.hexdigest("#{commit.author}:#{commit.message}")
# unseen will only be true if the id is not already in Redis
unseen = redis.setnx(id, Time.now.to_i)
# expire after 90 days to save on memory
redis.expire(id, 7776000) if unseen
# run notification if unseen
Or imagine you have a system that is applying regex to RSS feeds, and you want to be alerted at most once a day about matches:
id = Digest::MD5.hexdigest(regex_pat.to_s)
# Exclusively set the key, returning false if it already exists
unseen = redis.setnx(id, Time.now.to_i)
redis.expire(id, 86_400) if unseen
I do this in small scripts all the time. Now, I could write to sqlite, use the ruby PStore, or maybe a tempfile as lock, etc, but this just feels so dang simple, and I don’t worry about file locking, sql queries, weird commit code, etc.
Maybe you have scripts you want to “shut up” for a few hours. You could write a file and stat it, or you could just set a value to read:
# Don't bug for 5 minutes
redis.setex("shutup", 300, "true")
Often I have little tasks or cronjob on my home server that I want notifications about. Currently I use Pushover for all those notifications. Someday I might change my mind and switch to email or SMS. So what I’ve done is treat Redis like a good old Unix named pipe. But now I don’t have to worry about creating the pipe, or permissions, and I seem to remember Redis commands much easier (pop quiz…how do you call mkfifo in Ruby without shelling out?).
If you want to make a FIFO for notifications, write something like this:
# Producer script, cron job, etc
redis.rpush("notifications", "You've got mail!")
# Notification daemon
loop do
# blpop is a blocking read left pop
key, message = redis.blpop("notifications")
# send message to SMS, email, Pushover, etc
end
Sometimes, your notification daemon might die and not restart properly. Maybe you want to “catch up” on messages when you restart it, then the above list use is good. If you don’t want to get flooded by those missed notifications, simply switch to pub/sub:
# Producer
redis.publish("notifications", "A wild message arrives!")
# Notification daemon
Redis.new.subscribe("notifications") do |on|
on.message do |channel, msg|
send_notification(msg)
end
end
Now for one last tip. Redis provides redis-cli, a command line client for interacting with Redis. If you fire it up with no commands, you’ll get an interactive prompt. However, you can also invoke it with a command to run. This is great for shell scripts:
redis-cli publish notifications 'John sent you a private message on irc'
If you haven’t tried Redis yet and program small tasks, you can get started today!
—Jun 07, 2013
Shield is a simple authentication gem I tend to reach for first when developing small Sinatra or Cuba apps. The code is short, reliable, and easy to understand. It’s not a kitchen sink solution, it does one thing well: lets users log in with a password.
As an experiment, I wondered what it might look like to layer a simple two-factor authentication scheme on top of Shield. If you are unfamiliar with two-factor authentication, it typically follows that a user will first log in to a site with their password as usual, and if successful, must pass a second step of entering a PIN from a SecureID type token, or out-of-band delivery message such as SMS. Popular sites implementing two-factor auth include PayPal, Twitter, and Google.
Let’s break down some of the components of what a simple system might look like.
First, we’ll create a basic User model with Ohm, a persistance library backed by Redis.
class User < Ohm::Model
include Shield::Model
attribute :email
attribute :crypted_password
index :email
def self.fetch(email)
User.find(email: email).first
end
end
This is pretty straightforward. We’ve established a user with an email, crypted password, and the fetch method Shield expects per its implementation. Moving on, we’ll tweak the Shield helpers a little and setup some of our own for handling the second factor.
helpers do
include Shield::Helpers
alias_method :initially_authenticated, :authenticated
def authenticated(model)
if user = initially_authenticated(model)
user.id.to_s == session["#{model}_secondary_auth"].to_s && user
end
end
def challenge_authentication(model)
ChallengeAuthentication.new(initially_authenticated(model))
end
def send_challenge(model)
challenge_authentication(model).push
end
def challenge_accepted(model, challenge)
if challenge_authentication(model).check!(challenge)
user = initially_authenticated(model)
session["#{model}_secondary_auth"] = user.id.to_s
end
end
end
There’s a few interesting things to note here. First, we alias Shield’s authenticated method out to initially_authenticate. We’ll use this to check if a user passes the initial password authentication step. Next, we define our new authenticated method, which will rely on password authentication, and a second check against the session to see if the user has passed the secondary authentication step. Sprinkle in some methods around checking our challenge authentication (more details on that in a minute) and our helpers are good to go.
Now let’s move on to some simple Sinatra app and routing setup. First, we’ll handle 401 Unauthorized errors by redirecting to the /login path:
error 401 do
redirect '/login'
end
Next, add in the /login routes Shield typically expects, however, instead of redirecting on success to the main page, we’ll redirect the browser on to a verification step:
get '/login' do
erb :login
end
post '/login' do
if login(User, params[:login], params[:password])
remember(initially_authenticated(User)) if params[:remember_me]
send_challenge(User)
redirect '/login_verification'
else
redirect '/login'
end
end
Note that when a user passes the first login stage, we’ll send them the challenge for the verification step.
Now we’ll get into the meat of the routing and diverge from the vanilla Shield login flow a bit. Setup the verification handling:
get '/login_verification' do
error(401) unless initially_authenticated(User)
erb :login_verification
end
post '/login_verification' do
error(401) unless initially_authenticated(User)
if challenge_accepted(User, params[:challenge])
redirect '/'
else
redirect '/login_verification'
end
end
For either action to succeed, first the user must be initially authenticated. If so, we’ll verify the challenge presented, in this case a randomly assigned 6 digit PIN, matches up. Then we’ll let the user proceed on to the app.
The core of the challenge authentication is handled in a ChallengeAuthentication class:
class ChallengeAuthentication
EXPIRE_TIME = 300
attr_reader :user
def initialize(user)
@user = user
end
def redis
Ohm.redis
end
def push
challenge = generate_challenge
redis.setex key, EXPIRE_TIME, challenge
deliver_challenge(challenge)
end
def check(challenge)
return false if user.nil? || challenge.to_s.empty?
# Should be a secure compare to prevent timing attacks
redis.get(key) == challenge
end
def check!(challenge)
!! ( check(challenge) && redis.del(key) )
end
def key
[user.class.name, user.id, 'challenge'].join(':')
end
# Returns a 6 digit challenge phrase
def generate_challenge
(SecureRandom.random_number * 1_000_000).to_i
end
def deliver_challenge(challenge)
# send an out of band challenge like SMS or Pushover here
end
end
The flow of the challenge check is fairly simple. First, we can push a new challenge by setting a random 6 digit pin for the user, and deliver that out of band. This can be easily accomplished via SMS with a provider such as Twilio, or my favorite push notification service Pushover. Checkout my Rushover gem as a simple client for sending to Pushover. We’ll toss that PIN into Redis with a 5 minute expiration; this provides a simple limited window for which the PIN is valid. Likewise, if you want to implement this on top of a SQL ORM, you could add challenge and expiration timestamps on to your User model.
For checking that the user has provided a valid challenge PIN, we can compare against the value in Redis if it exists. Upon match, we’ll delete the PIN from redis to invalidate it and confirm the challenge is accepted.
You can find the complete code for this example as a gist. Hopefully it’s easy enough to follow, and now you can provide an extra level of security for your social cat-video apps!
Until next time…
—Jun 05, 2013
As a slight extension to the previous post, I wanted to make a quick point about authorization failures.
Given you’ve raised SomeAuthorizationFailure exception in a controller action, you might have a general rescue handling it:
rescue_from 'SomeAuthorizationFailure' do
render :text => "Bad user!", :status => 403
end
The key here is the 403 status, Forbidden. This is a pretty natural, and technically correct status to feed the client.
Cool, let’s wrap that up, it’s done! Hold on, not so fast.
If you use Github (if?), you may have noticed something that struck you as curious the first time it happened. Say you’re hanging in the dev campfire room, and somebody pastes a link to a line of code for you to checkout (like /foocorp/awesomeproject/config/application.rb#L7). You clicked on the link but forgot you’ve logged out. Boom….403 Forbidden.
Wait..no that’s not a 403, it’s a 404. What the heck?
The answer is pretty simple. On your little todo app you run for friends and family, it’s probably not a huge deal for somebody to hit /todo_lists/42/item/5 and get a 403. Wow, somebody now knows you have a todo list 42 and item 5. Probably not a big deal.
But on a site like Github, let’s change that application.rb link to say, /foocorp/awesomeproject/config/initializers/devise.rb. 403? Oh look, that project is using Devise!
The moral of the story: best to give a 404 status on authorization failures if you don’t want to cater to mining and leaking of sensitive info.
—Apr 24, 2013
Once upon a time, I’d heavily lean upon scoped finds for cheap authorization in Rails controller actions. For instance, a system might have many users, for which each has many projects they can manage. In order to find a project that a user can administer, an action may include the following:
@project = current_user.projects.find(params[:id])
This can work. If a user tries to hit project id 42, for which they aren’t associated with, the execution short circuits at that point. The security on that project has been maintained.
I think most people know at this point, this is a poor general authorization scheme, because for one, it spreads your authorization logic, no matter how simple, around the application. With a few controllers in a small system, this probably isn’t a big deal.
Enter an authorization scheme, you might write:
@project = current_user.projects.find(params[:id])
authorize @project
The authorize method here will typically take the current user, lookup some policy object, and run a check. If a user can be associated to a project, but not be able to edit it, this will probably pan out as you expect.
Sometimes though, you could simply write such code to be:
@project = Project.find(params[:id])
authorize @project
This can be a subtle, but I believe powerful, difference. First, your finder usage is simplified. But second, and I believe more importantly, the code becomes more straightforward and your exceptions more accurate. Take another look:
# If no project is found, raise ActiveRecord::RecordNotFound
@project = Project.find(params[:id])
# If the user is not authorized, raise SomeAuthorizationException
authorize @project
This is a worthwhile difference. Want to keep metrics or get alerted on security violation attempts? Now it can be clearly split. Or perhaps, you take different action or set different flash messages; this can be handled more cleanly now.
When it comes to patterns, remember, it’s never one-size-fits-all. What’s good to realize is that sometimes you can write your code in simpler fashions, and more importantly, think about the explicit exceptions your system should be throwing, if any.
—Apr 23, 2013
Often in a web application, the time will come where you opt to generate temporary passwords for users. One common approach to this is to use a helper that combines a small about of random data (such as 4 random numbers) with a word randomly selected from a pared down dictionary list.
Please don’t do this. There are at least two reasons:
- Using a word list radically reduces password entropy
- Eventually a password will unintentionally perplex, or worse, offend someone
If the day comes and the system generates the password “cigar1984” for somebody trying to quit smoking, that could be awkward.
Now, I’m no crypto expert, but I’m going to assume that the ruby SecureRandom library will do a better job than me at outputting random strings. So it’s of use here. For example:
require "securerandom"
# outputs something like "Ht25IeNqIBUp"
SecureRandom.urlsafe_base64.gsub(/[^a-z0-9]+/i, '')[0,12]
This strips non-alphanumeric characters such as ‘_’ and ‘-’ out. This is useful because people are less used to typing them, and also, certain mouseclick-to-copy behaviors will split on those characters depending on the environment.
It’s worth noting that certain characters often will confuse users if they are manually entering a password as viewed from the screen. Fonts can make characters including the following difficult to distinguish
- 0 (zero)
- 1 (the number one)
- I (the uppercase i)
- O (the uppercase OH)
- l (lowercase L)
You can String#tr these out for substitutions, or strip them alltogether. This will slightly reduce entropy, but by keeping a longer password you compensate somewhat.
—Jan 26, 2013