Table of Contents for

Opa: Up and Running

Imagine that we did not need all the information about some particular movie, but only the title of the movie with a particular ID. We could do that with the following query:

string title = /data/movies[{id: 1}]/title

The query /data/movies[{id: 1}] returns a single movie (with ID 1), and the remaining path, /title, means to project the resultant record to its single title field, which is of type string. Hence, that is the final type of such a query. It also works for queries with multiple results; for instance, to get the titles of all movies released in the year 2000, we could use the following query:

dbset(string, _) titles = /data/movies[release_year == 2000]/title

It is also possible to project into more than one field, although then, the syntax is slightly different. For example, imagine that we just wanted to fetch the title and the name of the director of a movie with a given ID; this query would do the job:

{string title, string director} m = /data/movies[{id: 1}].{title, crew.director}

Of course, it is also possible to do this kind of projection for multiple-result queries.

The main reason for using projections is performance. Most of the time it would be fine to fetch all the data from the database and only use the portions that we need. However, this may be an expensive operation, and we may be fetching a lot of information that we won’t use anyway. Projections allow us to fine-tune the information transfered from the database to our program.

You’ve already manipulated Birdy messages and introduced a type representing them, Msg.t. Now it is time to save them in the database for persistent storage. First, we will recapitulate the definition of the Msg.t type introduced in Modeling Messages:

abstract type Msg.t =
  { string content,
    User.t author,
    Date.date created_at
  }

Since this is a self-contained type with all the information about the message, including its author, content, and creation date, one possibility is to store all the Birdy messages as a set of values of that type. This can be accomplished with the following database declaration:

database msgs {
  Msg.t /all[{author, created_at}]
}

Here, we declare a primary key consisting of two fields: the author and the creation date of the message. Since dates work with millisecond precision, we assume that no author will publish two different messages in the same millisecond, and hence, this is a unique primary key.

The previous database declaration relates to messages, so we could just add it to the /src/model/msg.opa file. But as our strategy is to use a dedicated source file collecting all database declarations, we will slightly modify our declaration and add it to src/model/data.opa:

database birdy {
  User.info /users[{username}]
  Msg.t /msgs[{author, created_at}]
}

With the database declaration in place, we can now replace the dummy store function with a real one:

function void store(Msg.t msg) {
  /birdy/msgs[{author:msg.author, created_at:msg.created_at}] <- msg;
}

This function just adds a new entry to the /birdy/msgs set, indexed by the author and creation date of the given message.

exposed function void store(Msg.t msg) {
  /birdy/msgs[{author:msg.author, created_at:msg.created_at}] <- msg;
}

While developing code to render messages in Rendering Messages we introduced internal links of the shape /user/[USERNAME] and /topic/[TOPICNAME]. Those URLs will serve pages showing messages for a given user and topic, respectively. In order to develop such pages, we first need to fetch relevant messages that will be displayed on those pages; we will address this topic in this section and you will learn how to create those pages in User and Topic Pages.

What messages should be displayed on those pages? It is quite clear for the topics: every topic page should display all messages containing references to that topic. For the user pages, it is more complicated, as we want them to display:

First we’ll turn our attention to a function that returns all the messages for a given topic. How do we write it? Recall that our type for a message looks as follows:

abstract type Msg.t =
  { string content,
    User.t author,
    Date.date created_at
  }

The content contains the content of the message as an unstructured string and we were using the analyze function to decompose it into segments, with user and topic references. However, with this data organization we have no chance of performing our task effectively, as we would need to fetch all the messages, analyze them one by one, and filter those that mention the topic we are interested in, an approach that would quickly become unacceptable in terms of performance.

How can we improve it? By employing the classic technique of enriching the data with redundant information that will enable us to perform the data querying we need effectively. In our case, we need to know which users and which topics every message refers to, so the solution is to add two new fields containing this information to our Msg.t type:

abstract type Msg.t =
  { string content,
    User.t author,
    Date.date created_at,
    list(Topic.t) topic_refs,
    list(User.name) user_refs
  }

We now need to initialize those two fields in the create function that creates a new message. We would like to reuse the analyze function to get the list of topics and users referenced in the message, but the problem is that this function takes a Msg.t argument and we cannot supply it yet, as at this point we are in the process of creating a new message value. The solution is to change the type of this function to operate on the string containing the raw content of the message, so this:

function list(Msg.segment) analyze(Msg.t msg) {
  ...
  Parser.parse(msg_parser, msg.content)
}

becomes this:

private function list(Msg.segment) analyze_content(string msg) {
  [...]
  Parser.parse(msg_parser, msg)
}

function list(Msg.segment) analyze(Msg.t msg) {
  analyze_content(msg.content)
}

As you can see, we still make available the analyze function with the same type signature as before, which ensures that all the code outside of this module will work just as before. However, internally we develop a more low-level analyze_content function. We make it private to ensure that it is not visible from outside of the Msg module. We can now use it in the create function to initialize the topic_refs and user_refs fields:

function Msg.t create(User.t author, string content) {
  msg_segs = analyze_content(content)
  { ~content, ~author,
    created_at: Date.now(),
    topic_refs: get_all_topics(msg_segs),
    user_refs: get_all_users(msg_segs)
  }
}

We use two private functions, get_all_topics and get_all_users, that (given the list of segments of the message) return, respectively, the list of topics and users referenced in this message. A possible implementation of those functions could look as follows:

private function list(Topic.t) get_all_topics(list(Msg.segment) msg) {
  function filter_topics(seg) {
    match (seg) {
    case ~{topic}: some(topic)
    default: none
    }
  }
  List.filter_map(filter_topics, msg)
}

private function list(User.name) get_all_users(list(Msg.segment) msg) {
  function filter_users(seg) {
    match (seg) {
    case ~{user}: some(user)
    default: none
    }
  }
  List.filter_map(filter_users, msg)
}

Now we are done with user pages and topic fetching. It is easy to miss the importance of what happened here, though. Note that we changed the internal representation of messages in the system (by enriching it with some information) without making any changes outside of the message module. This was possible thanks to the fact that:

Having our message type enriched with data, we can now easily write functions to return messages related (in the aforementioned sense) to a given topic or user:

function msgs_for_topic(Topic.t topic) {
  /birdy/msgs[topic_refs[_] == topic; order -created_at; limit 50]
}

To understand this better, let’s take a look at all the components of this query and their meanings, step by step:

/msgs/all[                   
  topic_refs[_] == topic;    
  order -created_at;         
  limit 50                   
]

The function returning messages for a given user is only slightly more complicated:

function msgs_for_user(User.t user) {
  userdata = /birdy/users[{username: user.username}]       
  /birdy/msgs[author.username in userdata.follows_users or 
            topic_refs[_] in userdata.follows_topics or    
            user_refs[_] == user.username or               
            author.username == user.username;              
            order -created_at;                             
            limit 50]                                      
}

Now that we have functions to retrieve relevant messages, let’s construct user and topic pages. First, we need to take care of the navigation, or URL dispatching. This is the role of the controller, so we will add two more cases to the Controller.dispatcher function in /src/controller/main.opa:

function dispatcher(Uri.relative url) {
  match (url) {
  case {path: ["activation", activation_code] ...}:
    Signup.activate_user(activation_code)
  case {path:["user", user | _] ...}:
    Page.user_page(user)
  case {path:["topic", topic | _] ...}:
    Page.topic_page(topic)
  default:
    match (User.get_logged_user()) {
    case {~user}: Page.user_page(User.get_name(user))
    default: Page.main_page()
    }
  }
}

Note that we also modify default, adding a new case for the logged-in user that will display the user’s page upon signing in when the URL doesn’t change. This new dispatcher function needs to be connected to the Birdy User module, so we need to import birdy.model to the controller in the /src/opa.conf file:

birdy.controller:
  import birdy.{model,view}
  [...]

We’ll now turn our attention to the view in /src/view/page.opa, as we need to add two functions used—Page.topic_page and `Page.user_page`—that will construct pages for a given topic and user, respectively. Those two pages are similar in the sense that they display a list of messages, so let’s enclose this common feature in a function:

private function msgs_page(msgs, title, header) {
  msgs_iter = DbSet.iterator(msgs)
  msgs_html = Iter.map(MsgUI.render, msgs_iter)
  content =
    <div class=container>
      <div class=user-info>
        {header}
      </div>
      <div id=#msgs>
       {msgs_html}
      </div>
    </div>
  page_template(title, content, <></>)
}

Here, we’ve taken a list of database results as msgs and converted them to an iterator with DbSet.iterator. Then we converted those results to rendered messages using Iter.map with the MsgUI.render function. Finally, we built the HTML structure of our messages page to display messages.

With the messages page in place, we can easily build a page for a topic:

function topic_page(topic_name) {
  topic = Topic.create(topic_name)
  msgs = Msg.msgs_for_topic(topic)
  title = "#{topic}"
  header = <h3>{title}</h3>
  msgs_page(msgs, title, header)
}

Now we need a function to convert a string into a Topic.t type, which we place in the Topic module in /src/model/topic.opa:

function Topic.t create(string topic) {
  topic
}

We create a page for a user in a similar way, but if the requested user does not exist, we will display an error page:

function user_page(username) {
  match (User.with_username(username)) {
  case {some: user}:
    msgs = Msg.msgs_for_user(user)
    title = "@{username}"
    header = <h3>{title}</h3>
    msgs_page(msgs, title, header)
  case {none}:
    page_template("Unknown user: {username}", <></>,
      alert("User {username} does not exist", "error")
    )
  }
}

We need to add a function to get a user with a given username to the User module:

function option(User.t) with_username(string name) {
  ?/birdy/users[{username: name}] |> Option.map(mk_view, _)
}

And that’s it! By creating some messages and then clicking on the links contained in published messages or entering appropriate URLs by hand, you can verify that the pages work and display relevant messages.

Figure 10-2 is an example of a user page and Figure 10-3 is an example of a topic page.

Figure 10-2. Birdy user page

Figure 10-3. Birdy topic page

Now we would like to add one more micro-blogging feature to our Birdy app: the ability to follow other users and topics. Our plan is to:

The ability to follow will apply to logged-in users only, so first let’s write a function in the User module that will allow us to perform some action with logged-in users:

private function do_if_logged_in(action) {
    match (get_logged_user()) {
      case {guest}: void
      case {user: me}: action(me)
    }
}

Now let’s use the preceding function to write a function to follow a user:

function follow_user(user) {
  function mk_follow(me) {
    /birdy/users[{username: me.username}]/follows_users <+ user.username
  }
  do_if_logged_in(mk_follow)
}

The function mk_follow allows us to update the list. As you saw in Inserting/Updating Data, <+ is used to add an element at the end of the list. We then use the do_if_logged_in function to apply it only to logged-in users.

Using the same method, let’s write a function that allows us to unfollow a followed user:

function unfollow_user(user) {
  function mk_unfollow(me) {
    /birdy/users[username == me.username]/follows_users <-- [user.username]
  }
  do_if_logged_in(mk_unfollow)
}

Here we use <-- to remove the element from the list.

The next function we would like to write is a function that allows us to check if one given user A is followed by one logged-in user B:

function isFollowing_user(user) {
  match (get_logged_user()) {
  case {guest}: {unapplicable}
  case {user: me}:
    if (user.username == me.username) {
      {unapplicable}
    } else {
      if (/birdy/users[username == me.username and follows_users[_] == user.username]
          |> DbSet.iterator |> Iter.is_empty) {
        {not_following}
      } else {
        {following}
      }
    }
  }
}

This function applies to logged-in users and returns unapplicable for non-logged-in users as well as for the logged-in user B. The function looks for users who have the same username as logged-in user B and who follow given user A, and returns a DbSet that we transform to an iterator to check if the result is empty or not. If the result is empty, the function returns not_following; otherwise, it returns following.

Now we’ll use the same methods we just used to write the follow_topic, unfollow_topic, and isFollowing_topic functions:

function follow_topic(topic) {
  function mk_follow(me) {
    /birdy/users[{username: me.username}]/follows_topics <+ topic
  }
  do_if_logged_in(mk_follow)
}

function unfollow_topic(topic) {
  function mk_unfollow(me) {
    /birdy/users[username == me.username]/follows_topics <-- [topic]
  }
  do_if_logged_in(mk_unfollow)
}

function isFollowing_topic(topic) {
  match (get_logged_user()) {
  case {guest}: {unapplicable}
  case {user: me}:
    if (/birdy/users[username == me.username and follows_topics[_] == topic]
        |> DbSet.iterator |> Iter.is_empty) {
      {not_following}
    } else {
      {following}
    }
  }
}

The last thing we need to address is the Follow button. Let’s return to /src/view/page.opa and modify our msgs_page function as follows:

private function msgs_page(msgs, title, header, follow, unfollow, isFollowing) {
  recursive function do_follow(_) {
    _ = follow();
    #follow_btn = follow_btn();
  }
  and function do_unfollow(_) {
    _ = unfollow();
    #follow_btn = follow_btn();
  }
  and function follow_btn() {
    match (isFollowing()) {
    case {unapplicable}: <></>
    case {following}: <a class="btn" onclick={do_unfollow}>Unfollow</a>
    case {not_following}: <a class="btn btn-primary" onclick={do_follow}><i class="icon icon-white icon-plus"/> Follow</a>
    }
  }
  msgs_iter = DbSet.iterator(msgs)
  msgs_html = Iter.map(MsgUI.render, msgs_iter)
  content =
    <div class=container>
      <div class=user-info>
        {header}
        <div id=#follow_btn>{follow_btn()}</div>
      </div>
      {if (isFollowing() == {unapplicable} && Iter.is_empty(msgs_iter)) {
        <div class="well">
          <p>You don't have any messages yet. <a data-toggle=modal href="#{MsgUI.window_id}">Compose a new message</a>.</p>
        </div>
       } else <></>}
      <div id=#msgs>
       {msgs_html}
      </div>
    </div>
  page_template(title, content, <></>)
}

We add the do_follow and do_unfollow functions that take the user or topic and reconstruct the Follow button, and then we write the follow_btn function that takes the state returned by isFollowing and returns the corresponding HTML. We use Bootstrap classes to distinguish the Follow and Unfollow buttons by color. We create a <div> element with a #follow_btn identifier and call the follow_btn function. For a better user experience, we create a special page for the user (first-time) who doesn’t have any messages. We display a short notification about it and suggest that he create a new message. If the user has messages, they are displayed on the page.

Now let’s call the follow, unfollow, and isFollowing functions in the topic_page and user_page functions:

function topic_page(topic_name) {
  topic = Topic.create(topic_name)
  msgs = Msg.msgs_for_topic(topic)
  title = "#{topic}"
  header = <h3>{title}</h3>
  function follow() { User.follow_topic(topic) }
  function unfollow() { User.unfollow_topic(topic) }
  function isFollowing() { User.isFollowing_topic(topic) }
  msgs_page(msgs, title, header, follow, unfollow, isFollowing)
}

function user_page(username) {
  match (User.with_username(username)) {
  case {some: user}:
    msgs = Msg.msgs_for_user(user)
    title = "@{username}"
    header = <h3>{title}</h3>
    function follow() { User.follow_user(user) }
    function unfollow() { User.unfollow_user(user) }
    function isFollowing() { User.isFollowing_user(user) }
    msgs_page(msgs, title, header, follow, unfollow, isFollowing)
  case {none}:
    page_template("Unknown user: {username}", <></>,
      alert("User {username} does not exist", "error")
    )
  }
}

Compile and run your Birdy application to check the Follow button. Figure 10-4 shows a user’s page viewed by another logged-in user who doesn’t follow him. Figure 10-5 shows the topic page that is followed by the logged-in user: the Follow button has turned into an Unfollow button.

Figure 10-4. Following a Birdy user

Figure 10-5. Following a Birdy topic

This concludes our detailed tour of the functions we have implemented in Birdy. Now it’s time to play.