Paiza Engineering Blog

Engineering blog of browser-based web development environment PaizaCloud Cloud IDE ( https://paiza.cloud/ ), online compiler and editor Paiza.IO( https://paiza.IO/ )

Creating your PaizaCloud Cloud IDE Apps with HTML, JavaScript.

f:id:paiza:20171222165948p:plain
(Japanese article is here)

f:id:paiza:20151217152725j:plainHi, I'm Tsuneo(@).

PaizaCloud Cloud IDE is a browser-based Web development environment running in your browser, which can be used like Desktop OS(Windows, Mac) to casually develop Web applications.

As it runs on the browser, you don't need to do the cumbersome tasks like installation or setting up the development environment. You can just write code anywhere, anytime.

PaizaCloud have tools to operate the server, like file manager, text editor, terminal, or browser(browser-in-browser).

Those tools work for daily use. But, isn't it cool if you can add other tools like adding apps in PCs or smartphones.

So, here comes PaizaCloud Apps. By installing PaizaCloud Apps, you can add tools and features. You can install PaizaCloud Apps someone made. Or, you can create your own PaizaCloud App.

Isn't it fun if your friends use tools you made?

You can create the PaizaCloud Apps with HTML and JavaScript, just like Web pages.

Here, we'll create a simple drawing tool as an example PaizaCloud App.

Getting started with PaizaCloud Cloud IDE

Let's start!

Here is the website of PaizaCloud Cloud IDE.

https://paiza.cloud/

Just sign up with email and click a link in the confirmation email. You can also sign up with GitHub or Google.

Create new server

Next, let's create a new server for the development workspace.

f:id:paiza:20171214154558p:plain

Click "new server" to open a dialog to set up the server. Just click "New Server" button in the dialog without any settings.

f:id:paiza:20171219143410p:plain

Just in 3 seconds, you'll get a browser-based development environment.

How to use standard tools(Apps)

Before creating a new PaizaCloud App, let's see how standard tools work. Standard tools are listed on the left side of the page.

Upmost button with text "New file" is for a Text Editor. Click 'New file' button.

f:id:paiza:20171219143513p:plain

Now, we see the dialog box to set filename. Set the filename, and click 'Create' button.

f:id:paiza:20171218232517p:plain

After writing some text, click 'Save' button or type "Ctrl-S" or "Command-S" shortcut to save to the file. When you see the file finder on the left side of the page, you will also see the file "myapp.rb" you just created.

In these ways, you can load, edit and save text files. PaizaCloud has other tools like terminal or browser. Let's play with those tools.

Install PaizaCloud Apps

Now, let's see about PaizaCloud Apps.

PaizaCloud Apps is available as a button with text "Apps" and a PaizaCloud logo.

Create the PaizaCloud Apps button. You'll see the dialog to install PaizaCloud Apps.

f:id:paiza:20171222170153p:plain

Choose PaizaCloud App, and click "Install" button to install the PaizaCloud App.

Once installed, a button for the PaizaCloud App will appear on the left side of the page.

But for now, to create our own PaizaCloud app, let's click a button 'Create PaizaCloud App'.

Dialog to create PaizaCloud

You'll see a dialog box to create a PaizaCloud App.

f:id:paiza:20171222034107p:plain

Let's input the name of the application. You can also add an icon image, description, or extensions associated with the PaizaCloud App.

In the textarea, you can write codes using HTML or JavaScipt. You already have the template files to create a simple text editor. So, let's start with the template.

Creating PaizaCloud apps

Click 'save' button, and you'll see browser view for previewing the App.

f:id:paiza:20171222034247p:plain

Now, you can write HTML or JavaScript viewing the browser preview. (You can specify an external URL by clicking "External URL").

Let's change some title text and click 'Save' button. You'll get an updated browser preview right away.

PaizaCloud Apps JavaScript API

You cannot communicate with PaizaCloud to use features provided by PaizaCloud only by HTML or CSS. You'll need to use PaizaCloud JavaScript API to communicate with PaizaCloud.

You can call functions like file operation with this API.

PaizaCloud API uses a library Penpal.

To use the API, you need to connect PaizaCloud as below.

const connection = window.Penpal.connectToParent({methods: ...});

In the methods options, you can add function to handle calls from PaizaCloud. The following API functions are available.

API function feature
openFile(filename) On double click a file in the file finder

To use PaizaCloud features from the PaizaCloud app, you can use the "connection" object to call API functions. Following API functions are available.

API function feature
openSaveModal(options) Open modal dialog for saving.
openLoadModal(options) Open modal dialog for loading.
writeFile(file, data, options) Write data to a file.
readFile(file, data, options) Load data from a file.
execFile(file, args, options) Execute a file(program).

You can call those functions by adding the function name to the "connection" object. As you can get the result with Promise, you can call those functions like below.

connection.[function name](arguments).then((result) => { /* handle the result */});"

For about writeFile / readFile / execFile functions, the usage is the same as Node.js functions. (However, the result will be available as a Promise instead of a callback.)

In the template App, you can see those functions for reading and writing files in the "script.js". Check it out!

Creating a drawing App

Now, let's create a drawing App based on the template created.

In the HTML file, create canvas element to draw.

index.html:

<html>
<head>
  <link rel="stylesheet" href="style.css">
  <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
</head>
<body>
  <div class="header">
    <span>My Paint App</span>
    <button type="button" class="btn btn-default save">Save</button>
    <button type="button" class="btn btn-default load">Load</button>
  </div>
  <div class="main">
    <canvas class="absolute w-100 h-100" style="border: solid 1px black;"></canvas>
  </div>
  <script src="https://unpkg.com/penpal/dist/penpal.js"></script>
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
  <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
  <script src="script.js"></script>
</body>
</html>

Next, make a style sheet.

style.css:

body{
  width: 100%;
  height: 100%;
  display: flex;
  flex-direction: column;
}
.main{
  flex: 1 1 auto;
  position: relative;
}
.absolute{
  position: absolute;
}
canvas{
    border: solid 1px black;
    cursor: pointer;
}
.w-100{
  width: 100%;
}
.h-100{
  height: 100%;
}

Then, write JavaScript code. To handle mouse operation, add event listeners for "mousedown", "mousemove", or "mouseup" events. Draw to the canvas based on the mouse moves.

script.js:

// PaizaCloud App APIs:
//   Initialize:
//     * Penpal.connectToParent(options)
//   Call from App to PaizaCloud:
//     * openSaveModal(options)
//     * openLoadModal(options)
//     * writeFile(file, data, options, callback)
//     * readFile(file, data, options, callback)
//     * execFile(file, args, options, callback)
//   Call from PaizaCloud to App:
//     * openFile(filename)

var connection = window.Penpal.connectToParent({
  methods: {
    openFile: (filename) => {
      openFile(filename);
    }
  }
});
let parent;
connection.promise.then((parent_) => {
    parent = parent_;
    console.log('Penpal connected');
});

function openFile(filename) {
  parent.readFile(filename, {encoding: 'base64'}).then((result) => {
    var dataUrl = "data:image/png;base64," + result.data;
    var img = new Image();
    img.onload = function(){
        context.drawImage(img, 0, 0);
    }
    img.src = dataUrl;

  });
}
function saveFile(filename) {
    let dataUrl = canvas.toDataURL(); // data:image/png;base64,xxxxx
    let base64 = dataUrl.substr(22);
    parent.writeFile(filename, base64, {encoding: 'base64'});
}

$(".save").on('click', () => {
  parent.openSaveModal({extensions: ['txt']}).then((filename) => {
    saveFile(filename);
  });
});

$(".load").on('click', () => {
  parent.openLoadModal({extensions: ['txt']}).then((filename) => {
    openFile(filename);
  });
});

$('canvas').attr('width', $(".main").width());
$('canvas').attr('height', $(".main").height());

var canvas = $("canvas")[0];
var context = canvas.getContext('2d');

function getPos(event) {
    var x = event.clientX - canvas.getBoundingClientRect().x + canvas.scrollLeft;
    var y = event.clientY - canvas.getBoundingClientRect().y + canvas.scrollTop;
    return [x, y];
}
let lastPos;

$("canvas").on('mousedown', (event) => {
    lastPos = getPos(event);
});
$("canvas").on('mousemove', (event) => {
    if(!lastPos){return;}
    const p = getPos(event);
    context.beginPath();
    context.moveTo(lastPos[0], lastPos[1]);
    context.lineTo(p[0], p[1]);
    context.stroke();
    lastPos = p;
})
$("canvas").on('mouseup', (event) => {
    lastPos = null; 
});

That's it!

Click 'Save' button to save the App.

Install the drawing App

Now, let's install the App we just created.

Click 'My servers' to show the workspace page. Then, click "+ Apps" icon to open PaizaCloud App installation dialog box.

On the dialog box, choose the application created, and click 'Install' button.

After the installation, we'll see the App's icon on the left side of the page.

f:id:paiza:20171222034609p:plain

If you click the icon, you can launch the App! You can draw pictures, save the image to the file, or load the image from the file. Let's try!

f:id:paiza:20171222034806p:plain

f:id:paiza:20171222170325p:plain

Summary

We created a drawing PaizaCloud App and install it.

With PaizaCloud Apps, you can make your environment more convenient and comfortable.

As you can create the PaizaCloud App just with HTML and JavaScript, let's create Apps! Also, it would be more fun to share the PaizaCloud Apps with your friends or others!


With「PaizaCloud Cloud IDE」, you can flexibly and easily develop your Web application or server application, and publish it, just in your browser. https://paiza.cloud


Quick web development in browser using Ruby+Sinatra+PaizaCloud Cloud IDE!

f:id:paiza:20171219143149p:plain
(Japanese article is here)

f:id:paiza:20151217152725j:plainHi, I'm Tsuneo(@).

Everyone knows about Ruby on Rails, the Web application framework. sYes, Ruby on Rails is a major full-fledged MVC framework used in large-scale web applications.

At the same time, it is not very easy to use especially for beginners because there are many things to know in order to use Ruby on Rails.

Here comes Sinatra, yet another Ruby web application framework. Sinatra is a quite simple framework, so those who just learned Ruby can easily start writing.

When you start your first application, it is better to start with a small program, and then grow it step by step. With Sinatra, you can start with from 4 lines of code!

Well, it is another troublesome task to install and set up a Sinatra development environment and deploy it to the server. Because every PC has its own OS, application, and configuration, the setting up development environment often fails with errors.

Here comes PaizaCloud Cloud IDE, a browser-based web development environment. PaizaCloud Cloud IDE is quite flexible, and can be used not only for Ruby on Rails but also Sinatra. As both Sinatra and PaizaCloud Cloud IDE are simple and easy, it is the best combination for Ruby web development.

In this article, we'll actually develop a web service using Sinatra and PaizaCloud. We'll start with small program, and then create a web service that lookup a location from a host name, and shows it on the map.

By following the instructions, you'll create the application in about 10 minutes.

Getting started with PaizaCloud Cloud IDE

Let's start!

Here is the website of PaizaCloud Cloud IDE.

https://paiza.cloud/

Just sign up with email and click a link in the confirmation email. You can also sign up with GitHub or Google.

Create new server

Let's create a new server for the development workspace.

f:id:paiza:20171214154558p:plain

Click "new server" to open a dialog to set up the server. Just click "New Server" button in the dialog without any settings.

f:id:paiza:20171219143410p:plain

Just in 3 seconds, you'll get a browser-based development environment for Ruby on Rails.

Create application

Let's create a Sinatra application.

Create a Ruby file for Sinatra application. Here, we create a file "myapp.rb".

Click 'New file' button on the left side of the page.

f:id:paiza:20171219143513p:plain

Now, we see the dialog box to set filename. Set filename to "myapp.rb", and click 'Create' button.

f:id:paiza:20171219143628p:plain Now, let's create your first Sinatra code.

myapp.rb:

require 'sinatra'
get '/'
  'Hello ' + 'World'
end

f:id:paiza:20171218232517p:plain

After the coding, click 'Save' button or type "Ctrl-S" or "Command-S" shortcut to save to the file. When we see file finder on the left side of the page, we see a file "myapp.rb" that you just created.

Yes, it's a real web application with 4 lines of code.

Start Sinatra server

Then let's start the application.

To start the application, you need to run Ruby command. With PaizaCloud, you can use browser-based "Terminal" to run commands.

Click 'Terminal' button on the left side of the workspace.

f:id:paiza:20171214154805p:plain

Now, the Terminal started. Type commands like "ruby myapp.rb" and type enter key.

$ ruby myapp.rb

f:id:paiza:20171218232646p:plain

On the left side of workspace, you'll see a button with earth icon and "4567".

f:id:paiza:20171218232727p:plain

Sinatra server runs on port 4567. PaizaCloud Cloud IDE detects the port number(4567), and automatically adds the button to open a browser for the port.

Click the button, and you'll get Browser application(a browser application in the PaizaCloud). Now, you'll see web page with "Hello World".

f:id:paiza:20171219000404p:plain

Show HTML file

Next, instead of one message 'Hello World', let's create and show a HTML file.

Change "myapp.rb" to show HTML file as below. We use ERB template format to write Ruby code in the HTML file. We'll load "sinatra/reloader" to reload ruby code from the file after changing the code automatically.

myapp.rb:

require 'sinatra'
require 'sinatra/reloader'

get '/' do
    erb :index
end

Then, create a HTML(ERB) file.

On the left side of the workspace, right-click the home directory("/home/ubuntu"), and select "New Directory" to create a directory with name "views".

Then right-click the "views" directory, and select "New File", and create a file with filename "index.erb".

index.erb:

<h1>Host/IP Map</h1>

At the Terminal, type Ctrl-C to exit the running Sinatra application. Then, type "ruby myapp.rb" to restart Sinatra. As we load 'sinatra/reloader', we won't need to restart Sinatra from now.

Now, let's reload the Browser(in PaizaCloud). You'll see the HTML file created!

Install a package(gem)

We install a library to lookup locations from IP addresses. Ruby have a bunch of gem package, and we can add many features to the program. For now, we'll install a gem package called "maxminddb". We also download a database for the lookup.

Run following commands in the Terminal.

$ sudo gem install maxminddb
$ curl http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz -O
$ gunzip GeoLite2-City.mmdb.gz

f:id:paiza:20171218235045p:plain

At PaizaCloud, you can install whatever additional packages like this!

Show location, map

With maxminddb gem, let's lookup the location from the host's IP address. Also, show the map of the location using Google Maps.

Change the code("myapp.rb") like below. At "post" block, we can refer the hostname with field name "host" as "params['host']". Then, lookup IP address from the hostname, lookup location using the maxminddb, and set it to a instance variable "@geo". Instance variables like "@geo" can be referred from the ERB file.

Also, load "erubis" library to escape HTML automatically.

myapp.rb:

require 'sinatra'
require 'sinatra/reloader'
require 'maxminddb'
require 'erubis'
set :erb, :escape_html => true

db = MaxMindDB.new('./GeoLite2-City.mmdb')

get '/' do
    erb :index
end
post '/' do
    host = params['host']
    @ip = IPSocket::getaddress(host)
    @geo = db.lookup(@ip)
    erb :index
end

Change the ERB file. Add input tag to input a hostname or an IP address. When the form is submitted, show the contents of @geoip. <% if @geoip %> ... <% end %> means, the HTML inside the "if" block is shown only when the variable "@geoip" is available.

Also, show a map using iframe and Google Maps.

(We use Google Maps Embed API. Get your API key from here , and put it after "key=" of the URL.)

Also, let's add a CSS framework Milligram to make the HTML beautiful.

views/index.erb:

<html>
<head>
    <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Roboto:300,300italic,700,700italic">
    <link rel="stylesheet" href="//cdn.rawgit.com/necolas/normalize.css/master/normalize.css">
    <link rel="stylesheet" href="//cdn.rawgit.com/milligram/milligram/master/dist/milligram.min.css">
</head>
<body>
    <div class="container">
    <h1>IP&Host Map</h1>
    <form method="post">
        <input type="text" name="host" value="<%= params['host'] %>" placeholder="www.example.com or 127.0.0.1">
        <input type="submit">
    </form>

    <% if @geo %>
        <table>
            <tr>
                <td>IP</td>
                <td><%= @ip %></td>
            </tr>
            <tr>
                <td>Country</td>
                <td><%= @geo.country.name %></td>
            </tr>
            <tr>
                <td>Subdivision</td>
                <td><%= @geo.subdivisions[0].name %></td>
            </tr>
            <tr>
                <td>City</td>
                <td><%= @geo.city.name %></td>
            </tr>
        </table>
        <iframe src="https://www.google.com/maps/embed/v1/place?q=<%= @geo.location.latitude %>,<%= @geo.location.longitude %>&zoom=10&key=AIzaSyAAb1Nx1f7E6uNrEEiukZV5z9XX3TqkDNw" 
    width="600" height="300" allowfullscreen="allowfullscreen"></iframe>
        (<div>This product includes GeoLite2 data created by MaxMind, available from
<a href="http://www.maxmind.com">http://www.maxmind.com</a>.</div>)
    <% end %>
</body>    
</html>

That's it! You did it!

Let's open the web application in the browser(in PaizaCloud). You see the location and the map of the hostname you put!

(If you get an error, try to restart Sinatra. Exit "ruby myapp.rb" by "Ctrl-C", and run the command again.)

f:id:paiza:20171218235726p:plain

Summary

With PaizaCloud Cloud IDE, we created a Sinatra application just in your browser, without installing or setting up any development environments. The combination of Sinatra and PaizaCloud is one of the fastest and shortest way to create web application. Now, let's create your own Sinatra application!


With「PaizaCloud Cloud IDE」, you can flexibly and easily develop your web application or server application, and publish it, just in your browser. https://paiza.cloud


How to create Ruby on Rails app in browser with PaizaCloud Cloud IDE

f:id:paiza:20171214155211p:plain
(Japanese article is here)

f:id:paiza:20151217152725j:plainHi, I'm Tsuneo([twitter:@yoshiokatsuneo]).

When starting Ruby on Rails, what's the most difficult thing? Is it Ruby syntax, object-oriented programming, or web technology? Actually, the hardest thing is installing and setting up the development environment!

You'll bump into issue like these:

  • No instruction for the latest version released today.
  • Installation error. (Ex: Nokogiri...)
  • The requirement of another software or libraries on the platform.
  • Instructions that works only on Mac, but not on Windows.
  • Conflicts with other software installed.
  • Finally, it is installed and configured. But other software was broken because of this.
  • etc.

You can ask your friends, but they have already lost how to install, or they are using older versions, or they have customized things too much...

Also, if you publish the service, feedbacks from your friends or others will motivate you. But, this requires "deployment" of the service. The "deployment" also frustrates us...

So, here comes PaizaCloud Cloud IDE. PaizaCloud Cloud IDE enables you to develop web services using Ruby on Rails just in your browser, without installing and setting up any development environment. Actually, you can create your own service in 5 minutes, and even publish it on the Internet to share it with friends.

Getting started with PaizaCloud Cloud IDE

So, here is the website of PaizaCloud Cloud IDE.

https://paiza.cloud/

Just sign up with email and click a link in the confirmation email. You can also sign up with GitHub or Google.

Create new server

Let's create a new server for the development workspace.

f:id:paiza:20171214154558p:plain

Click "new server" to open a dialog to set up the server.

Here, you can choose "Ruby on Rails", "phpMyAdmin", and "MySQL", and click "New Server" button.

f:id:paiza:20171214154330p:plain

Just in 3 seconds, you'll get a browser-based development environment for Ruby on Rails.

Create an application

Then, let's create your Ruby on Rails application.

You can use "rails new" command to create Ruby on Rails application.

On PaizaCloud Cloud IDE, you can use PaizaCloud's "Terminal" application to run the commands in your browser.

Let's click the "Terminal" button at the left side of the page.

f:id:paiza:20171214154805p:plain

Now, the "Terminal" application launch. So, let's type "rails new [application name]" command in the Terminal.

f:id:paiza:20171213234417p:plain

The '[applicatio name]' is the name of the application you are creating. You can choose whatever you want, like "music-app" or "game-app".

Here, I'll choose the application name "boardgame-app", where I can manage the list of board games.

Also, let's add "--database=mysql" to use MySQL database.

So, lets type:

$ rails new boardgame-app --database=mysql

In the file finder view at the left side of the page, you'll see the "boardgame-app" directory. Click the folder to open it to see inside the directory.

f:id:paiza:20171213234518p:plain

You'll see a bunch of files for the Ruby on Rails application.

You'll already have a MySQL server running because you checked it on the server setting. But if not, you can always manually start like:

$ sudo systemctl enable mysql
$ sudo systemctl start mysql

On PaizaCloud Cloud IDE, you can install packages on root privilege.

Then, create a database for the application. Change current directory to the application directory, and type "rake db:create".

$ cd boardgame-app
$ rake db:create

f:id:paiza:20180922200831p:plain

A database "boardgame-app_development" for the application is created.

Start Ruby on Rails server

Now, you can already run the application. Let's start the application.

Change the directory by "cd boardgame-app", and type "rails server" to start the server!

$ cd boardgame-app
$ rails server

f:id:paiza:20171213234757p:plain

You'll get a new button with text "3000" on the left side of the page.

f:id:paiza:20171213234820p:plain

Ruby on Rails server runs on port 3000. PaizaCloud Cloud IDE detects the port number(3000), and automatically adds the button to open a browser for the port.

Click the button, and you'll get Browser application(a Browser application in the PaizaCloud). Now, you'll see web page about Ruby on Rails, that is your application!

f:id:paiza:20171213234947p:plain

Create new database table

Next, let's use database on the application.

On Ruby on Rails, you can just use "rails generate scaffold" command to create all files or a database table, a model, a controller, or a routing.

Here, we create a table with "name" fields for the name of the boardgame, and "player" fields for the number of players.

Now, open another terminal(or, type 'Ctrl-C' to exit "rails server") and type:

$ rails generate scaffold game name players

f:id:paiza:20181207141345p:plain

After that, run commands for creating and migrating database table.

$ rake db:create
$ rake db:migrate

f:id:paiza:20171213235143p:plain

Done! Put "http://localhost:3000/games/" in the URL bar of the PaizaCloud browser application.

Then, you'll get the list of the board game. Let's try to add or delete the board game information.

f:id:paiza:20171213235249p:plain

You can see the database table using phpMyAdmin.

On PaizaCloud browser application, put "http://localhost/phpmyadmin/" in the URL bar.

f:id:paiza:20171214003801p:plain

Edit files

Now, let's change the title by editing the file.

To edit a file, double-click the file on the file finder view.

Open "boardgame-app/app/views/games/index.html.erb" for editing.

f:id:paiza:20171213235414p:plain

Then, change the title inside <h1>.

boardgame-app/app/views/games/index.html.erb:

<h1>My Board Games!</h1>

Then, click "Save" button, or push "Command-S", or "Ctrl-S".

f:id:paiza:20171213235606p:plain

Let's add an image file to "boardgame-app/app/assets/images" directory. You can drag and drop a file on your PC for uploading.

f:id:paiza:20181207133959p:plain

Edit "index.html.erb" to show the image file.

boardgame-app/app/views/games/index.html.erb:

<h1>My Board Games!</h1>
<%= image_tag 'boardgame.jpg' , width: '100', height: '100' %>

To see the list from top page, let's redirect from top page to the "/games/" page.

Open "config/route.rb", and append routing for the root page like below.

config/route.rb:

  root to: redirect('/games/')

f:id:paiza:20171214000215p:plain

Now, the basics of the application is done!

f:id:paiza:20171214155455p:plain

Uploading image

Then, let's add a feature to uploading an image file for the board game.

On Ruby on Rails, you can install "gem" packages to add features like this.

Here, to use the "paperclip" gem for uploading files, add "gem 'paperclip'" to a file called "Gemfile".

Gemfile:

gem 'paperclip'

f:id:paiza:20171214000430p:plain

Then, add "picture" field for uploaded files to the "game" table by running commands:

~/boardgame-app$ bundle install
~/boardgame-app$ rails generate paperclip game picture

Now, you have a migration file like "~/boardgae-app/db/migrations/201xxxxx_add_attachment_picture_to_games.rb".

Add [5.2] to the end of "ActiveRecord::Migration".

~/boardgae-app/db/migrations/201xxxxx_add_attachment_picture_to_games.rb:

class AddAttachmentPictureToGames < ActiveRecord::Migration[5.2]

f:id:paiza:20171214000553p:plain

Run migration to apply to the database.

$ rake db:migrate

To manage images, add tags to HTML files for posting form and showing data.

~/boardgame-app/app/views/games/_ form.html.erb:

  <%= form.file_field :picture %>

~/boardgame-app/app/views/games/index.html.erb

        <td><%= image_tag game.picture.url(:thumb) %></td>

Add a field "picture" to the controller.

~/boardgame-app/app/controllers/games_controller.rb

    def game_params
      params.require(:game).permit(:name, :players, :picture)
    end

Change the model to manage uploaded files.

app/models/game.rb

class Game < ApplicationRecord
    has_attached_file :picture, styles: { medium: "300x300>", thumb: "100x100>" }, default_url: "/images/:style/missing.png"
    do_not_validate_attachment_file_type :picture
end

Adding CSS framework

To make the page beautiful, let's add a minimalist CSS framework Milligram.

app/views/layouts/aplication.html.erb

  <head>
    <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Roboto:300,300italic,700,700italic">
    <link rel="stylesheet" href="//cdn.rawgit.com/necolas/normalize.css/master/normalize.css">
    <link rel="stylesheet" href="//cdn.rawgit.com/milligram/milligram/master/dist/milligram.min.css">

f:id:paiza:20171214001352p:plain

Now, it's done! Isn't it cool!?

f:id:paiza:20171214155235p:plain

Summary

With PaizaCloud Cloud IDE, we created a Ruby on Rails application just in your browser, without installing or setting up any development environments. Now, let's create your own Ruby on Rails application!


With「PaizaCloud Cloud IDE」, you can flexibly and easily develop your web application or server application, and publish it, just in your browser. https://paiza.cloud


Using Docker is not risky. But, there are things to look out for.

f:id:paiza:20160607145104p:plain

(Japanese article is here.)

f:id:paiza:20140712194904j:plain (by Yoshioka Tsuneo, @ at https://paiza.IO/)

Docker is rapidly growing because it is a simple and easy-to-use lightweight virtual environments tool. But, because Docker has its own concept, and is adding more and more features, it is sometimes not very easy to have correct information and it may cause misunderstandings.

Especially, security risks tend to be overestimated or underestimated based emotion rather than on accurate information. But, it is vital to know accurate information to use Docker as a handy and secure tool.

So, here, I describe Docker container security, and things to look out for.

Myths that using Docker is risky.

Docker host can be compromised by just running container.

Because containers run on isolated environments, just running the containers(without options) does not expose Docker host to the containers. As of now, there is no known case that just running containers allows malicious containers to invade Docker host or other containers.

If you share your Docker host's directories or files with the container, the container can access the Docker host. But, the host's directories or files are not shared with the container unless you explicitly share them. (ex: by -v option)

Also, unless you explicitly open the containers' network ports to Docker host(by -p, -P option), no one outside of Docker host can connect to the container.

Note that, compared to virtual machines, containers are much more tightly coupled with operating systems. So, there is relatively more possibility to have unknown vulnerabilities and attackers abuse the vulnerability , especially when you run the containers as a root user. Always update to the latest version or apply security fixes, and pay attention to the latest security news. Consider running the container as a non-root user or using user namespace. It also makes sense to install security software.

Container's root user and Docker host's root user is always the same

In the past, there was no way to have separate user id namespace for containers, and the container and the host share the same root user.

From ver 1.10, we can use user namespace. A Docker host and containers on the host can have separate root users. Although there are multiple barriers between the containers and the host, because the root user can do so many things, it is possible to have holes to access the Docker host. Using a non-root user on the containers makes Docker container much secure.

Docker shouldn't allow containers to access host anyway

Containers that manage Docker system itself apparently need to access Docker host to manage the Docker system. For example, Shipyard(container management tool) or nginx-proxy(setting proxy by hooking container events) need to access Docker system.

Docker containers are vulnerable to the fork bomb.

In the past, there was no direct way to protect from fork bomb.

But, Docker 1.11 introduced a "--pid-limits" option which limits the number of processes on the container to protect from the fork bomb**.

Things to look out for

While there is no need to fear overly to use Docker, there are things to take care for container security.

Running container as Internet server

If you run a container as an Internet server, you should look out for security as with a normal server. Set the appropriate access control, and pay attention to the server and the application's security issues.

Never run an untrusted Docker image as an Internet server, and avoid running Docker images you don't fully understand as an Internet server.

Docker does not open container's network ports unless you explicitly open the ports. When you run Docker host on a virtual machine, the containers are not open to the Internet (unless the virtual machine is open to the Internet). If you use Docker on your PC(Windows, Mac), its firewall or router will protect the containers from connecting from the Internet(unless you explicitly allow it).

Docker command options(sharing files, opening ports)

Pay attention to the Docker command options.

Especially, if you set file sharing option(ex: -v), you allow the container to access Docker host. Take care when you share socket files such as "docker.sock" that allow the container to manage Docker. Don't share files too many, but share only necessary files with the containers.

If you open containers' network ports, and the Docker host is open to the Internet, then the container can be accessed from the Internet. If you don't have appropriate access control, attackers may intrude on your containers. When you open containers' network ports, set the appropriate access control.

If the Docker image is not stable, consider not to open network ports or explicitly disable network connection(--net=none).

Running unknown scripts

Apart from Docker, never run unknown scripts because it is quite risky. The risk is not only intrusions to the container or Docker hosts. It can be a malicious script that removes all your files or install viruses, worms, or trojans.

Pay attention to how you get the script, whether the site providing the script is reliable or not, and whether the script is downloaded using HTTPS(SSL). You can also read the scripts.

Running unknown containers

Malicious or problematic containers may abuse Internet access, or unnecessarily consume CPU or memory resource.

Official images on famous registries like Docker Hub have relatively less risky images. But, unknown containers provided by unknown sites will be high-risk.

Evaluate the risk of Docker images, and when necessary, disable network access or limit access to the resources.

Summary

Docker is a simple and easy-to-use tool that has both good parts and risk.

Especially, when you open the container's network ports to the Internet, pay attention as normal Internet server.

Docker gives you freedom to create and destroy environments instantly. It is not risky if you use reliable Docker images and does not open the container's network ports to the Internet.

Don't fear too much, but use the Docker understanding its concept, behavior, and usage.

P.S. For Japanese readers:
I wrote a Japanese book for Docker(Docker実戦活用ガイド). (An article for the book.)


paiza.IO is an online coding environment where you can just write and run code instantly. Just try it out!



Building a Q&A web service in an hour - MEAN stack development(3)

(Japanese article is here.)

f:id:paiza:20140712194904j:plain (by Yoshioka Tsuneo (@))

MEAN stack (*) is an all-in-one JavaScript-based web service development environment supporting front-end, back-end, and database development. A MEAN stack environment, AngularJS Full-Stack generator, provides the best practice to develop clean software quickly.

(*) MEAN stack packs MongoDB, Express, AngularJS, and Node.js.

In the first article, I introduced how to install the MEAN stack. In the second article, I introduced how to build Twitter-like web service.

In this article, as an example of a more practical web service, I introduce how to build a Q&A service like Stack Overflow, Qiita, or even Reddit or HackerNews. The way can be used for features like Blog or SNS comments where users can comment, discuss, or communicate each other.

In the second article (Twitter-like service), we build a service with one page. In this Q&A service, we will build multiple pages using generator. We also use input validations using validators.

The Q&A service has the following features:

  • Listing questions
  • Create, edit, or delete questions
  • Create, edit, or delete answers for each question
  • Create, edit, or delete comments on each question or answer
  • Tags for each question
  • Markdown editing
  • Searching questions, answers, or comments
  • Staring questions, answers, or comments
  • Listing all questions, my questions, or starred questions

The Q&A service is available in the following URL: http://paizaqa.herokuapp.com

Source code is available below: https://github.com/gi-no/paizaqa

So, let's build the Q&A service!

Contents

Installing MEAN stack

If you have not installed, install a MEAN stack, Angular Full-Stack generator. See Installation section in the first article for installation instruction.

Confirm that installed AngularJS Full-Stack generator is ver3.3.0 or later.

$ npm ls -g generator-angular-fullstack
/usr/local/lib
└── generator-angular-fullstack@3.3.0 

If it is older than ver3.3.0, update to the latest version.

$ sudo npm update -g generator-angular-fullstack

Generating a new project

At first, we generate a project from templates. Let's name the project "paizaqa" for now. We use "yo" Yeoman command to generate the project.

% mkdir paizaqa
% cd paizaqa
% yo angular-fullstack paizaqa

We use almost all default settings but enable SNS authentication (oAuth).

- Would you like to include additional oAuth strategies? 
 ◉ Google
 ◉ Facebook
❯◉ Twitter

Start the project.

% grunt serve

A browser will open the generated project on http://localhost:9000/ .

Directory structure

The generated project has the following structure.

.
|-- bower.json                            Bower packages (Client-side libraries)
|-- package.json                          npm packages (Server-side libraries)
|
|-- client                                Client-side codes
|   |-- app
|   |   |-- app.js                        Client-side main JavaScript code
|   |   `-- main
|   |       |-- main.controller.js        Client-side controller code
|   |       |-- main.controller.spec.js   Client-side test code
|   |       |-- main.html                 HTML template file
|   |       |-- main.js                   Client-side routing configuration
|   |       `-- main.scss                 CSS file
|   |-- components
|   |   |-- navbar
|   |   |   |-- navbar.controller.js      Navbar controller
|   |   |   `-- navbar.html               Navbar HTML template file
|   |   `-- socket
|   |       `-- socket.service.js         Client-side WebSocket code
|   `-- index.html
|
`-- server                                Server-side code
    `-- api
        `-- thing
            |-- index.js                  Server-side API routing configuration
            |-- thing.controller.js       Server-side controller (API implementation)
            |-- thing.model.js            Server-side DB model
            |-- thing.socket.js           Server-side WebSocket implementation
            `-- thing.integration.js      Server-side test code

Client-side codes are deployed under a client directory, and server-side codes are deployed under a server directory. By packaging files into a directory for each feature as a component, components become independent of each other and the whole project is clean and easy to understand.

On client directory, each "app/MODULE" directory stores files for each URL routing as a component. The main files for directories are HTML files (ex: "main.html") and client-side controllers (ex: "main.controller.js"). Other files are client-side routing configurations, test codes, CSS files, etc. Common features for the project are stored under the "components" directory as a subdirectory.

On server directory, each "server/api/MODULE" directory stores files for each URL routing as a component. The main files for the directories are Database models and server-side controllers. Other files are server-side routing configurations, server-side WebSocket implementations, and test codes.

The client and the server send or receive data or events by communicating client-side controllers and server-side controllers using JSON-based HTTP APIs. From MVC model's perspective, the server sees the client as views, and the client sees the server as models.

Creating server-side question component (model, API, etc.)

In this Q&A service, each question is stored as a document in a database.

Generate server-side question-related directory and files (DB mode, server-side controller, etc.) using the generator.

When prompted for a endpoint, set the default ("/api/questions"). The generator generates "server/api/question" directory with files such as "question.controller.js", and "question.model.js". The "/api/question" API is prepared for use.

% yo angular-fullstack:endpoint question

Next, we edit the database model to store question titles, question contents, and the list of answers. MongoDB can directly store arrays or associated arrays as a part of one JSON object. MongoDB itself is flexible schema and does not require pre-defined schemas. But mongoose driver used in Angular Full-Stack generator provides schema as an additional feature to limit or validate fields. So, we define question-related information as a schema.

server/api/question/question.model.js

var QuestionSchema = new mongoose.Schema({
  title: String,
  content: String,
});

Also, edit test code to adjust edited model. Just replace all the "name" and "info" with "title" and "content", respectively (5 places).

server/api/question/question.integration.js

// name: ...
title: ...
// info: ...
content: ...
...
// newQuestion.name....
newQuestion.title....
// question.info....
newQuestion.content....
...
// question.name....
question.title....
// question.info....
question.content....
...
// name: ...
title: ...
// info: ...
content: ...
...
// updatedQuestion.name....
updatedQuestion.title....
// question.info....
updatedQuestion.content....

Creating client-side question-listing, question-creating, and question-showing components (controllers, HTMLs, etc.)

Now, we create files for question-listing, question-creating, and question-showing components (controllers, HTMLs).

Removing needless files

Remove needless files from the project.

% rm -r client/app/main

Generating components

Generate question-related directories and files. Now, we create three directories (questionsIndex, questionsCreate, and questionsShow) for question listing, question-creating, and question showing.

When generator prompted for URL routing as "What will the url of your route be?", type the following URLs.

  • questionsIndex (Question listing): /
  • questionsCreate (Question creating): /questions/create
  • questionsShow (Question showing): /questions/show/:id
% yo angular-fullstack:route questionsIndex
? Where would you like to create this route? client/app/
? What will the url of your route be? /
% yo angular-fullstack:route questionsCreate
? Where would you like to create this route? client/app/
? What will the url of your route be? /questions/create
% yo angular-fullstack:route questionsShow
? Where would you like to create this route? client/app/
? What will the url of your route be? /questions/show/:id

Directories and files are generated. Now, we implement client-side controllers and HTMLs by editing the files.

Editting question-listing routing

To make question-listing page main page, set state for the question-listing page to "main".

client/app/questionsIndex/questionsIndex.js:

      .state('main', {

Editing question-listing controller

On the question-listing controller, we retrieve question listing using "GET /api/questions" API. Store the retrieved question to $scope variable so that we can refer from the HTML file. Because we use "$http" service, add "$http" to the controller function parameters. The service based on the parameter variable name is assigned to the parameter.

client/app/questionsIndex/questionsIndex.controller.js

  .controller('QuestionsIndexCtrl', function ($scope, $http) {
    $http.get('/api/questions').success(function(questions) {
      $scope.questions = questions;
    });
  });

Editing question-listing HTML file

On the question-listing HTML file, we can refer to "$scope.question" as "questions". By writing an attribute as 'ng-repeat="question in questions"', we can repeatedly output the elements for each question. Also, we can refer to "$scope" variable such as "{{question.title}}".

client/app/questionIndex/questionIndex.html

<header class="hero-unit" id="banner">
  <div class="container">
    <h1>paizaQA</h1>
    <p class="lead">Kick-start your next web app with Angular Fullstack</p>
    <img src="assets/images/yeoman.png" alt="I'm Yeoman">
  </div>
</header>

<div class="container">
  <br/>
  <div style="text-align: center">
    <a type="button" class="btn btn-primary" href="/questions/create">Ask Question</a>
  </div>

  <table class="table table-striped">
    <thead>
      <tr>
        <th>Question</th>
      </tr>
    </thead>
    <tbody>
      <tr ng-repeat="question in questions">
        <td>
          <a ng-href="/questions/show/{{question._id}}" style="font-size: large">{{question.title}}</a>
        </td>
      </tr>
    </tbody>
  </table>
</div>

client/app/questionIndex/questionIndex.scss

#banner {
    border-bottom: none;
    margin-top: -20px;
}

#banner h1 {
    font-size: 60px;
    line-height: 1;
    letter-spacing: -1px;
}

.hero-unit {
    position: relative;
    padding: 30px 15px;
    color: #F5F5F5;
    text-align: center;
    text-shadow: 0 1px 0 rgba(0, 0, 0, 0.1);
    background: #4393B9;
}

Editing question crating controller

Implement "$scope.submit()" function called when questions are submitted. The function stores the submitted question ($scope.question) to the server using "POST /api/questions" API. After the submission, move to submission-listing page using "$location.path('/questions')".

client/app/questionsCreate/questionsCreate.controller.js

...
  .controller('QuestionsCreateCtrl', function ($scope, $http, $location) {
    $scope.submit = function() {
      $http.post('/api/questions', $scope.question).success(function(){
        $location.path('/');
      });
    };
  });

Editing question-creating HTML file

On the question-creating HTML file, add "ng-submit" attribute to call "submit()" on submissions. Add 'ng-model="question.title"' attribute to input tag to synchronize between input tag and "question.title" variable bi-directionally.

client/app/questionsCreate/questionsCreate.html

<div class="container">
  <form name="form" ng-submit="submit()">
    <h2>Title:</h2>
    <input type="text" class="form-control" ng-model="question.title">
    <br>
    <h2>Question:</h2>
    <textarea rows=10 cols=80 ng-model="question.content"></textarea>
    <input type="submit" class="btn btn-primary" value="Post question">
  </form>
</div>

Editing question-showing controller

On the question-showing controller, retrieve question contents and show it. Question ID can be retrieved from URL("/question/show/:id") by referring to ":id" part as "$stateParams.id".

client/app/questionsShow/questionsShow.controller.js

  .controller('QuestionsShowCtrl', function ($scope, $http, $stateParams) {
    var loadQuestions = function(){
      $http.get('/api/questions/' + $stateParams.id).success(function(question) {
        $scope.question = question;
      });
    };
    loadQuestions();
  });

Editing question-showing HTML file

On the question-showing HTML file, output question title and contents. We refer to $scope.question" variable set on controller as "question".

client/app/questionsShow/questionsShow.html

<div class="container" id="question-show-container">
  <div>
    <div>
      <h1>{{question.title}}</h1>
    </div>
  </div>
  <hr/>
  {{question.content}}
</div>

client/app/questionsShow/questionsShow.scss

#question-show-container .comment{
  hr {
    margin: 0;
  };
  p {
    margin: 0;
  };
  margin-left: 100px;
};

Restart the service

Generally, we don't need to restart the service. However, because we changed the directory structure dramatically, we restart the service to make sure that our changes to code are applied.

% grunt serve

Creating answer field

Now, though we are building the Q&A service, we can only ask questions and no one can answer them. So, let's enable to create and display answers.

Editing server-side DB model

Edit QuestionSchema to store answers. MongoDB can store JSON object including arrays into a document (corresponding to a record in RDB).

server/api/question/question.model.js

var QuestionSchema = new mongoose.Schema({
  title: String,
  content: String,
  answers: [{
    content: String,
  }],
});

Editing server-side routing

Add answer submission API to the URL routing.

server/api/question/index.js

router.post('/:id/answers', controller.createAnswer);

Editing server-side controller

Implement answer submission API. We can add a value to an array by using MongoDB's '$push' operator.

server/api/question/question.controller.js

function createAnswer(req, res) {
  Question.update({_id: req.params.id}, {$push: {answers: req.body}}, function(err, num) {
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
};

Editing client-side question-showing controller

Add "$scope.submitAnswer()" function called on answer submission. The function sends the answer to the server using "POST /api/questions/QUESTION-ID/answers" API. Reload the whole question after the submission.

client/app/questionsShow/questionsShow.controller.js

    ...
    loadQuestions();
    $scope.newAnswer = {};
    $scope.submitAnswer = function() {
      $http.post('/api/questions/' + $stateParams.id + '/answers', $scope.newAnswer).success(function(){
        loadQuestions();
        $scope.newAnswer = {};
      });
    };

Editing client-side question-showing HTML

Output the list of answers stored in the question. Add "ng-submit" attribute to call "$scope.submitAnswer()" on answer submission.

client/app/questionsShow/questionsShow.html

  ...
  {{question.content}}
  &nbsp;
  <h3>{{question.answers.length}} Answers</h3>
  <div ng-repeat="answer in question.answers">
    <hr/>
    <div class="answer">
      {{answer.content}}
    </div>
  </div>
  <hr/>
  <h3>Your answer</h3>
  <form name="answerForm" ng-submit="submitAnswer()">
    <textarea rows=10 cols=80 ng-model="newAnswer.content"></textarea>
    <input type="submit" class="btn btn-primary" value="Submit your answer">
  </form>
</div>

client/app/questionsShow/questionsShow.css

...
#question-show-container .answer{
  margin-left: 50px;
};

Using Markdown

For now, we can use only plain text for questions or answers. Let's support Markdown like used in Stack Overflow.

We can just add a module and edit tags to support Markdown.

Installing angular-pagedown module

Install angular-pagedown module to support Markdown. When prompted for AngularJS version, choose an option to use the latest version.

% bower install angular-pagedown --save
...
Unable to find a suitable version for angular, please choose one:
    1) angular#~1.2 which resolved to 1.2.28 and is required by angular-pagedown#0.4.3
    2) angular#>=1.2.* which resolved to 1.4.3 and is required by paizaqa
    3) angular#>= 1.0.8 which resolved to 1.4.3 and is required by angular-ui-router#0.2.15
    4) angular#>=1 which resolved to 1.4.3 and is required by angular-bootstrap#0.11.2
    5) angular#^1.2.6 which resolved to 1.4.3 and is required by angular-socket-io#0.6.1
    6) angular#1.4.3 which resolved to 1.4.3 and is required by angular-resource#1.4.3

Prefix the choice with ! to persist it to bower.json

? Answer: 2

Add lines to "bower.json" to load depending files.

bower.json

{
  ...
  },
  "overrides": {
    "pagedown": {
      "main": [
        "Markdown.Converter.js",
        "Markdown.Sanitizer.js",
        "Markdown.Extra.js",
        "Markdown.Editor.js",
        "wmd-buttons.png"
      ]
    }
  }
}

Adding angular-pagedown module to application

To make use of the module, add "ui.pagedown" to application depending modules.

client/app/app.js

angular.module('paizaqaApp', [
  ... ,
  'ui.pagedown',
])

Using pagedown tag

Enable Markdown input. Change from the "textarea" tag to the "pagedown-editor" tag, and set the bound variable by using "content" attribute. For markdown output, change from "{{}}" to "pagedown-viewer" tag.

client/app/questionsCreate/questionsCreate.html

<!-- <textarea ... ng-model="question.content">...</textarea> -->
<pagedown-editor ng-model="question.content"></pagedown-editor>

client/app/questionsShow/questionsShow.html

<!-- {{question.content}} -->
<pagedown-viewer content="question.content"></pagedown-viewer>
...
<!-- {{answer.content}} -->
<pagedown-viewer content="answer.content"></pagedown-viewer>
...
<!-- <textarea ... ng-model="newAnswer.content"></textarea> -->
<pagedown-editor ng-model="newAnswer.content"></pagedown-editor>

Adding question tags

To make it easy to understand kinds of questions, let's add tags related to questions (ex: "Android", "Objective-C") to each question.

Editing database model

Edit QuestionSchema to store tags as an array.

server/api/question/question.model.js

var QuestionSchema = new mongoose.Schema({
  title: String,
  content: String,
  answers: [{
    content: String,
  }],
  tags: [{
    text: String,
  }],
});

Installing ngTagsInput module

Install ngTagsInput module to make it easy to edit or show tags.

% bower install ng-tags-input --save

Adding ngTagsInput module to the application modules

To make use of ngTagsInput module, add the module to application depending modules.

client/app/app.js

angular.module('paizaqaApp', [
  ...
  'ngTagsInput',
])

Editing question-creating HTML file

Add question tags input field using "tags-input" tag. You can also add auto completion by adding "auto-complete" element inside "tags-input" element, but for simplicity, we'll omit the auto completion this time.

client/questionsCreate/questionsCreate.html

    <pagedown-editor ng-model="question.content"></pagedown-editor>
    <h2>Tags:</h2>
    <tags-input ng-model="question.tags">
      <!-- <auto-complete source="loadTags($query)"></auto-complete> -->
    </tags-input>

Editing question-listing HTML file

Add tags below the question title.

client/app/questionsIndex/questionsIndex.html

          <a ng-href="/questions/show/{{question._id}}" style="font-size: large">{{question.title}}</a>
          <div>
            <span ng-repeat="tag in question.tags">
              <span class="label label-info">
                {{tag.text}}
              </span>
              &nbsp;
            </span>
          </div>

Editing question-showing HTML file

Add tags below the question title.

client/app/questionsShow/questionsShow.html

      <h1>{{question.title}}</h1>
      <span ng-repeat="tag in question.tags">
        <span class="label label-info">
          {{tag.text}}
        </span>
      </span>

User authentication

For now, we don't know who submits questions and answers. So, let's add user authentication feature. Only submitted users can edit or remove the articles. Also, store the submission date.

Editing server-side database model

Store submitted user's object ID to questions and answers. Specify object's referring model as "ref: 'User'" so that "populate()" function can expand user IDs to User objects.

Though we can manually call pupulate() from each query, in this project, to exapand User object for all the query, hook "find()" and "findOne()" call using "pre()" and call pupulate() in the handler. "populate('user')" can expands all the fields. But in this project, write "pupulate('user', 'name')" to exapand only 'name' field of the user object.

server/api/question/question.model.js

var QuestionSchema = new mongoose.Schema({
  title: String,
  content: String,
  answers: [{
    content: String,
    user: {
      type: mongoose.Schema.ObjectId,
      ref: 'User'
    },
    createdAt: {
      type: Date,
      default: Date.now,
    },
  }],
  tags: [{
    text: String,
  }],
  user: {
    type: mongoose.Schema.ObjectId,
    ref: 'User'
  },
  createdAt: {
    type: Date,
    default: Date.now
  },
});
QuestionSchema.pre('find', function(next){
  this.populate('user', 'name');
  this.populate('answers.user', 'name');
  next();
});
QuestionSchema.pre('findOne', function(next){
  this.populate('user', 'name');
  this.populate('answers.user', 'name');
  next();
});

Editing server-side API routing

On the server-side API routing, add "auth.isAuthenticated()" as an Express middleware to the URL resource requiring authentication so that the server-side controller can refer to the current login user as "req.user". Also, add an answer-deleting API(DELETE /:id/answers/:answerId).

server/api/question/index.js

var auth = require('../../auth/auth.service');

router.get('/', controller.index);
router.get('/:id', controller.show);
router.post('/', auth.isAuthenticated(), controller.create);
router.put('/:id', auth.isAuthenticated(), controller.update);
router.patch('/:id', auth.isAuthenticated(), controller.update);
router.delete('/:id', auth.isAuthenticated(), controller.destroy);

router.post('/:id/answers', auth.isAuthenticated(), controller.createAnswer);
router.put('/:id/answers/:answerId', auth.isAuthenticated(), controller.updateAnswer);
router.delete('/:id/answers/:answerId', auth.isAuthenticated(), controller.destroyAnswer);

Editing server-side controller

Change the query to return the last 20 questions. "sort({createdAt: -1})" sort by created time in descending order and "limit(20)" returns the first 20 objects. After creating the query, call "execAsync()" to execute the query.

server/api/question/question.controller.js

export function index(req, res) {
  Question.find().sort({createdAt: -1}).limit(20).execAsync()
    ...

Change the question-creating API to save a user as a part of question.

server/api/question/question.controller.js

export function create(req, res) {
  req.body.user = req.user;
  Question.create(req.body, ...
...
export function createAnswer(req, res) {
  req.body.user = req.user;
  Question.update(...

On question-updating and question-destroying API, verify that current login user ID is the same as the question's user ID so that only submitted users can edit or delete the articles.

server/api/question/question.controller.js

function handleUnauthorized(req, res) {
  return function(entity) {
    if (!entity) {return null;}
    if(entity.user._id.toString() !== req.user._id.toString()){
      res.send(403).end();
      return null;
    }
    return entity;
  }
}
...
// Updates an existing Question in the DB
export function update(req, res) {
  ...
    .then(handleEntityNotFound(res))
    .then(handleUnauthorized(req, res))
    ...
...
// Deletes a Question from the DB
export function destroy(req, res) {
  ...
    .then(handleEntityNotFound(res))
    .then(handleUnauthorized(req, res))
    ...

Implement answer-destroying API. Delete the answer specified by answer's ID and answer's user ID using MongoDB '$pull' operator.

export function destroyAnswer(req, res) {
  Question.update({_id: req.params.id}, {$pull: {answers: {_id: req.params.answerId , 'user': req.user._id}}}, function(err, num) {
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
};

Implement answer-updating API. On the MongoDB query, we can refer the matching index of the array (answer array in this case) as '$'. Add a condition to match current login user and answer's user ID so that only submitted user can update the answer.

export function updateAnswer(req, res) {
  Question.update({_id: req.params.id, 'answers._id': req.params.answerId}, {'answers.$.content': req.body.content, 'answers.$.user': req.user.id}, function(err, num){
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
};

Editing client-side question-listing HTML

Add user name and created time to the question.

client/app/questionsIndex/questionsIndex.html

          <a ng-href="/questions/show/{{question._id}}" style="font-size: large">{{question.title}}</a>
          <div class="clearfix"></div>
          <div style="float: right;">
            by {{question.user.name}}
             · {{question.createdAt}}
          </div>

Editing client-side question-showing controller

Implement the question- and answer-deletion functions ("deleteQuestion()", "deleteAnswer()") and updating functions ("updateQuestion()", "updateAnswer()"). The deleting functions call "DELETE /api/questions/:id" and "DELETE /api/questions/:id" API. The updating functions call "PUT /api/questions/:id" and "PUT /api/questions/:id/answers/:id" API.

Also, implement "isOwner()" function which checks whether the user of question or answer matches the current login user. We can use "Auth.isLoggedIn()" to check whether the user is logged in, and "Auth.getCurrentUser()._id" to get current login user ID. To user "$location" and "Auth" modules, add "$location" and "Auth" parameter to the controller function. AngularJS automatically assigns the services to the parameters from the parameter names.

client/app/questionsShow/questionsShow.controller.js

angular.module('paizaqaApp')
  .controller('QuestionsShowCtrl', function ($scope, $http, $stateParams, Auth, $location) {
    ...
    $scope.deleteQuestion = function() {
      $http.delete('/api/questions/' + $stateParams.id).success(function(){
        $location.path('/');
      });
    };
    $scope.deleteAnswer = function(answer) {
      $http.delete('/api/questions/' + $stateParams.id + '/answers/' + answer._id).success(function(){
        loadQuestions();
      });
    };  
    $scope.updateQuestion = function() {
      $http.put('/api/questions/' + $stateParams.id, $scope.question).success(function(){
        loadQuestions();
      });
    };
    $scope.updateAnswer = function(answer) {
      $http.put('/api/questions/' + $stateParams.id + '/answers/' + answer._id, answer).success(function(){
        loadQuestions();
      });
    };
    $scope.isOwner = function(obj){
      return Auth.isLoggedIn() && obj && obj.user && obj.user._id === Auth.getCurrentUser()._id;
    };

Editing client-side question-showing HTML file

Add submitted user name and created time to the question and answers.

client/app/questionsShow/questionsShow.html

  <pagedown-viewer content="question.content"></pagedown-viewer>
  <div class="text-right">by {{question.user.name}} · {{question.createdAt}}</div>
...
    <pagedown-viewer content="answer.content"></pagedown-viewer>
    <div class="text-right">by {{answer.user.name}} · {{answer.createdAt}}</div>

Also, add delete button if current login user is the same as question or answer user. Use "isOwner()" function to check user, and set "ng-click" attribute to call deleteQuestion/deleteAnswer() on deletion.

client/app/questionsShow/questionsShow.html

    <button ng-if="isOwner(question)" type="button" class="close" ng-click="deleteQuestion()">&times;</button>
    <div>
      <h1>{{question.title}}</h1>
...
    <button ng-if="isOwner(answer)" type="button" class="close" ng-click="deleteAnswer(answer)">&times;</button>
    <pagedown-viewer content="answer.content"></pagedown-viewer>

Also, add editing button. "editing" variable is true while editing. Use "ng-show"/"ng-if" to switch element by "editing" variable. Although we are using the same "editing" variable for the question and answers, answers are under an element with the "ng-repeat" attribute and "ng-repeat" create separate scope for each item. So, each "editing" actually refers to different variables for the question and each answer.

client/app/questionsShow/questionsShow.html

      <h1>
        <div ng-if="! editing">{{question.title}}</div>
        <input type=text ng-model="question.title" ng-if=" editing">
      </h1>
      ...
  <pagedown-viewer content="question.content" ng-if="!editing"></pagedown-viewer>
  <pagedown-editor ng-model="question.content" ng-if=" editing"></pagedown-editor>
  <button type="submit" class="btn btn-primary" ng-click="editing=false;updateQuestion()" ng-show=" editing">Save</button>
  <a ng-click="editing=!editing;" ng-show="isOwner(question) && !editing">Edit</a>
  ...
    <div class="answer">
      ...
      <pagedown-viewer content="answer.content" ng-if="!editing"></pagedown-viewer>
      <pagedown-editor ng-model="answer.content" ng-if=" editing"></pagedown-editor>
      <button type="submit" class="btn btn-primary" ng-click="editing=false;updateAnswer(answer)" ng-show=" editing">Save</button>
      <a ng-click="editing=!editing;" ng-show="isOwner(answer) && !editing">Edit</a>
    ...

Editing client-side question-creating controller

If the current user is not authenticated, move to the login page.

client/app/questionsCreate/questionsCreate.controller.js

  .controller('QuestionsCreateCtrl', function ($scope, $http, $location, Auth) {
    if(! Auth.isLoggedIn()){
      $location.path('/login');
      $location.replace();
      return;
    }
    ...

Editing server-side test

In this project, remove routing test.

% rm server/api/question/index.spec.js

For APIs requiring authentication, before each test, login and set authentication information before the test.

server/api/question/question.integration.js

var User = require('../user/user.model');
...
describe('Question API:', function() {
  var user;
  before(function() {
    return User.removeAsync().then(function() {
      user = new User({
        name: 'Fake User',
        email: 'test@test.com',
        password: 'password'
      });

      return user.saveAsync();
    });
  });

  var token;
  before(function(done) {
    request(app)
      .post('/auth/local')
      .send({
        email: 'test@test.com',
        password: 'password'
      })
      .expect(200)
      .expect('Content-Type', /json/)
      .end(function(err, res) {
        token = res.body.token;
        done();
      });
  });
  ...    
  describe('POST /api/questions', function() {
    ...    
        .post('/api/questions')
        .set('authorization', 'Bearer ' + token)
    ...
  describe('PUT /api/questions/:id', function() {
    ...
        .put('/api/questions/' + newQuestion._id)
        .set('authorization', 'Bearer ' + token)
    ...
  describe('DELETE /api/questions/:id', function() {
    ...
        .delete('/api/questions/' + newQuestion._id)
        .set('authorization', 'Bearer ' + token)
    ...
        .delete('/api/questions/' + newQuestion._id)
        .set('authorization', 'Bearer ' + token)
    ...
  /* describe('PUT /api/things/:id', function() {
  }); */

Input validation

For now, we can submit a form with an empty question, or an empty answer. Let's validate inputs not to submit without filling the field.

valication

Installing ngMessage module

Install ngMessage module for the validation as a client-side library.

% bower install angular-messages --save

Adding ngMessage module to application module

Add the ngMessage module to the application depending modules to use the module.

client/app/app.js

angular.module('paizaqaApp', [
  ...
  'ngMessages',
])

Editing client-side HTML files

Add validations (ex: "required") to the input fields. Also, to refer the validation results, add name (with "name" attribute) to the form and add name(with "name" attribute) and model (with "ng-model" attribute) to each input field. The validation result is stored as "FORM-NAME.FIELD-NAME.$error", and output the result using ng-messages/ng-message attributes in which only matching elements are shown. The whole form validation result is stored as "FORM-NAME.$invalid", and we can disable the submit button when invalid.

client/app/questionsCreate/questionsCreate.html

  <form name="form" ng-submit="submit()">
    <h2>Title:</h2>
    <input type="text" class="form-control" ng-model="question.title" name="question_title" required>
    <span class="text-danger" ng-messages="form.question_title.$error">
      <span ng-message="required">Required</span>
    </span>
    <span class="text-success" ng-show="form.question_title.$valid">OK</span>
    <br>
    <h2>Question:</h2>
    <pagedown-editor ng-model="question.content" ng-model="question.content" name="question_content" required></pagedown-editor>
    <span class="text-danger" ng-messages="form.question_content.$error">
      <span ng-message="required">Required</span>
    </span>
    <span class="text-success" ng-show="form.question_content.$valid">OK</span>
    <h2>Tags:</h2>
    <tags-input ng-model="question.tags">
      <!-- <auto-complete source="loadTags($query)"></auto-complete> -->
    </tags-input>
    <input type="submit" class="btn btn-primary" ng-disabled="form.$invalid" value="Post question">
  </form>

client/app/questionsCreate/questionsShow.html

    <pagedown-editor ng-model="newAnswer.content" name="answerEditor" required></pagedown-editor>
    <input type="submit" class="btn btn-primary" ng-disabled="answerForm.$invalid" value="Submit your answer">

Time formatting filter

For now, the time format for created time is UTC. Let's change it to show the time from now like in Stack Overflow.

Installing Moment.js library

Install Moment.js library for time formatting as a client-side library.

% bower install --save momentjs

Add a "moment-with-locales.min.js" file for I18N (no need for English).

client/index.html

    <!-- build:js({client,node_modules}) app/vendor.js -->
      <!-- bower:js -->
      ...
      <!-- endbower -->
      <script src="bower_components/moment/min/moment-with-locales.min.js"></script>
      <script src="socket.io-client/socket.io.js"></script>
    <!-- endbuild -->

Generating filter

Generate a filter boilerplate.

% yo angular-fullstack:filter fromNow
% grunt injector

Implementing filter

Format time using the Moment.js's fromNow() function. You can optionally use locale() function to set language.

client/app/fromNow/fromNow.filter.js

    return function (input) {
      return moment(input).locale(window.navigator.language).fromNow();
    };

Using filter

Change time format from UTC to time from now using the fromNow filter we created. To use filter, change from "{{EXPRESSION}}" to "{{EXPRESSION|FILTER}}". In this case, we change from "{{EXPRESSION}}"を"{{EXPRESSION|fromNow}}".

client/app/questionsIndex/questionsIndex.html

<!-- Old: {{question.createdAt}} -->
{{question.createdAt|fromNow}}

client/app/questionsCreate/questionsShow.html

<!-- Old: {{question.createdAt}} -->
{{question.createdAt|fromNow}}
<!-- Old: {{answer.createdAt}} -->
{{answer.createdAt|fromNow}}

Changing test code

Fix the failing test.

Change test code to test that the fromNow filter with the current time(Date.now()) returns 'a few seconds ago'.

client/app/fromNow/fromNow.filter.spec.js

  it('return "a few seconds ago" for now', function () {
    expect(fromNow(Date.now())).toBe('a few seconds ago');
  });

Adding comments

Now, let's add comment fields for questions and answers.

Editing server-side database model

On QuestionSchema, store comments as an array inside the each question and answer. Each comment holds the created time and submitted user.

server/api/question/question.model.js

var QuestionSchema = new mongoose.Schema({
  ...
  answers: [{
    ...
    comments: [{
      content: String,
      user: {
        type: mongoose.Schema.ObjectId,
        ref: 'User'
      },
      createdAt: {
        type: Date,
        default: Date.now,
      }
    }],
  }],
  ...
  comments: [{
    content: String,
    user: {
      type: mongoose.Schema.ObjectId,
      ref: 'User'
    },
    createdAt: {
      type: Date,
      default: Date.now,
    }
  }],
});

QuestionSchema.pre('find', function(next){
  this.populate('user', 'name');
  this.populate('comments.user', 'name');
  this.populate('answers.user', 'name');
  this.populate('answers.comments.user', 'name');
  next();
});
QuestionSchema.pre('findOne', function(next){
  this.populate('user', 'name');
  this.populate('comments.user', 'name');
  this.populate('answers.user', 'name');
  this.populate('answers.comments.user', 'name');
  next();
});

Editing server-side routing

Add the following APIs to create, update, and delete a comment on a question or an answer.

Method URL Description
POST /:id/comments Create a comment on a question
PUT /:id/comments/:commentId Update a comment on a question
DELETE /:id/comments/:commentId Delete a comment on a question
POST /:id/answers/:answerId/comments Create a comment on an answer
PUT /:id/answers/:answerId/comments/:commentId Update a comment on an answer
DELETE /:id/answers/:answerId/comments/:commentId Delete a comment on an answer

server/api/question/index.js

router.post('/:id/comments', auth.isAuthenticated(), controller.createComment);
router.put('/:id/comments/:commentId', auth.isAuthenticated(), controller.updateComment);
router.delete('/:id/comments/:commentId', auth.isAuthenticated(), controller.destroyComment);

router.post('/:id/answers/:answerId/comments', auth.isAuthenticated(), controller.createAnswerComment);
router.put('/:id/answers/:answerId/comments/:commentId', auth.isAuthenticated(), controller.updateAnswerComment);
router.delete('/:id/answers/:answerId/comments/:commentId', auth.isAuthenticated(), controller.destroyAnswerComment);

Editing server-side controller

On the question-listing API, expand a user ID of a comment to a user object using populate().

server/api/question/question.controller.js

exports.show = function(req, res) {
  Question.findById(req.params.id).populate('user', 'name').populate('comments.user', 'name').populate('answers.user', 'name').populate('answers.comments.user', 'name').exec(function (err, question) {
    ...

Implement APIs to create, update, or delete a comment. To create a comment, add a comment to the comment array of the question document using '$push' operator. To delete a comment, delete a comment from the comment array of the question document. To update a comment, specify the updating index of the comment array of the question document using '$'. To update a comment on an answer, because we update an item inside an array of an array and only one '$' can be used to specify the index, we need to iterate each item for the array.

server/api/question/question.controller.js

/* comments APIs */
export function createComment(req, res) {
  req.body.user = req.user.id;
  Question.update({_id: req.params.id}, {$push: {comments: req.body}}, function(err, num){
    if(err) {return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  })
}
export function destroyComment(req, res) {
  Question.update({_id: req.params.id}, {$pull: {comments: {_id: req.params.commentId , 'user': req.user._id}}}, function(err, num) {
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
}
export function updateComment(req, res) {
  Question.update({_id: req.params.id, 'comments._id': req.params.commentId}, {'comments.$.content': req.body.content, 'comments.$.user': req.user.id}, function(err, num){
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
}

/* answersComments APIs */
export function createAnswerComment(req, res) {
  req.body.user = req.user.id;
  Question.update({_id: req.params.id, 'answers._id': req.params.answerId}, {$push: {'answers.$.comments': req.body}}, function(err, num){
    if(err) {return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  })
}
export function destroyAnswerComment(req, res) {
  Question.update({_id: req.params.id, 'answers._id': req.params.answerId}, {$pull: {'answers.$.comments': {_id: req.params.commentId , 'user': req.user._id}}}, function(err, num) {
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
}
export function updateAnswerComment(req, res) {
  Question.find({_id: req.params.id}).exec(function(err, questions){
    if(err) { return handleError(res)(err); }
    if(questions.length === 0) { return res.send(404).end(); }
    var question = questions[0];
    var found = false;
    for(var i=0; i < question.answers.length; i++){
      if(question.answers[i]._id.toString() === req.params.answerId){
        found = true;
        var conditions = {};
        conditions._id = req.params.id;
        conditions['answers.' + i + '.comments._id'] = req.params.commentId;
        conditions['answers.' + i + '.comments.user'] = req.user._id;
        var doc = {};
        doc['answers.' + i + '.comments.$.content'] = req.body.content;
        /*jshint -W083 */
        Question.update(conditions, doc, function(err, num){
          if(err) { return handleError(res)(err); }
          if(num === 0) { return res.send(404).end(); }
          exports.show(req, res);
          return;
        });
      }
    }
    if(!found){
      return res.send(404).end();
    }
  });
}

Editing client-side controller

Add functions to add, update, or delete a comment that send the request to the server.

client/app/questionsShow/questionsShow.controller.js

    $scope.newComment = {};
    $scope.submitComment = function() {
      $http.post('/api/questions/' + $stateParams.id + '/comments', $scope.newComment).success(function(){
        loadQuestions();
        $scope.newComment = {};
        $scope.editNewComment = false;
      });
    };
    $scope.submitAnswerComment = function(answer) {
      $http.post('/api/questions/' + $stateParams.id + '/answers/' + answer._id + '/comments', answer.newAnswerComment).success(function(){
        loadQuestions();
      });
    };
    $scope.deleteComment = function(comment) {
      $http.delete('/api/questions/' + $stateParams.id + '/comments/' + comment._id).success(function(){
        loadQuestions();
      });
    };
    $scope.deleteAnswerComment = function(answer, answerComment) {
      $http.delete('/api/questions/' + $stateParams.id + '/answers/' + answer._id + '/comments/' + answerComment._id).success(function(){
        loadQuestions();
      });
    };
    $scope.updateComment = function(comment) {
      $http.put('/api/questions/' + $stateParams.id + '/comments/' + comment._id, comment).success(function(){
        loadQuestions();
      });
    };
    $scope.updateAnswerComment = function(answer, answerComment) {
      $http.put('/api/questions/' + $stateParams.id + '/answers/' + answer._id + '/comments/' + answerComment._id, answerComment).success(function(){
        loadQuestions();
      });
    };

Editing client-side question-showing HTML file

Output comments' content, created time, and user for the question or the answers in the question object. Also, add forms to submit new comments.

client/app/questionsShow/questionsShow.html

  <div class="text-right">by <a ng-href="/users/{{question.user._id}}">{{question.user.name}}</a> · {{question.createdAt|fromNow}}</div>
  &nbsp;
  <div class="comment">
    <div ng-repeat="comment in question.comments">
      <hr/>
      <button ng-if="isOwner(comment)" type="button" class="close" ng-click="deleteComment(comment)">&times;</button>

      <pagedown-viewer content="comment.content" ng-if="!editing"></pagedown-viewer>
      <pagedown-editor ng-model="comment.content" ng-if=" editing"></pagedown-editor>
      <button type="submit" class="btn btn-primary" ng-click="editing=false;updateComment(comment)" ng-show=" editing">Save</button>
      <a ng-click="editing=!editing;" ng-show="isOwner(comment) && !editing">Edit</a>

      <div class="text-right" style="vertical-align: bottom;">by <a ng-href="/users/{{comment.user._id}}">{{comment.user.name}}</a> · {{comment.createdAt|fromNow}}</div>
      <div class="clearfix"></div>
    </div>
    <hr/>
    <a ng-click="editNewComment=!editNewComment;">add a comment</a>
    <form ng-if="editNewComment" name="commentForm">
      <pagedown-editor ng-model="newComment.content" editor-class="'comment-wmd-input'"
        ng-model="newComment.content" name="commentEditor" required>
      </pagedown-editor>
      <button type="button" class="btn btn-primary" ng-click="submitComment()" ng-disabled="commentForm.$invalid">Add Comment</button>
    </form>
  </div>
  ...
  <h3>{{question.answers.length}} Answers</h3>
  <div ng-repeat="answer in question.answers">
    ...
    <div class="text-right">by {{answer.user.name}} · {{answer.createdAt|fromNow}}</div>
    <div class="comment">
      <div ng-repeat="comment in answer.comments">
        <hr/>
        <button ng-if="isOwner(comment)" type="button" class="close" ng-click="deleteAnswerComment(answer, comment)">&times;</button>
 
        <pagedown-viewer content="comment.content" ng-if="!editing"></pagedown-viewer>
        <pagedown-editor ng-model="comment.content" ng-if=" editing"></pagedown-editor>
        <button type="submit" class="btn btn-primary" ng-click="editing=false;updateAnswerComment(answer, comment)" ng-show=" editing">Save</button>
        <a ng-click="editing=!editing;" ng-show="isOwner(comment) && !editing">Edit</a>

        <div class="text-right">by <a ng-href="/users/{{question.user._id}}">{{comment.user.name}}</a> · {{comment.createdAt|fromNow}}</div>
        <div class="clearfix"></div>
      </div>
      <hr/>
      <a ng-click="editNewAnswerComment=!editNewAnswerComment;answer.newAnswerComment={}">add a comment</a>
      <form ng-if="editNewAnswerComment" name="answer_{{answer.id}}_comment">
        <hr/>
        <pagedown-editor ng-model="answer.newAnswerComment.content" editor-class="'comment-wmd-input'"
          ng-model="answer.newAnswerComment.content" required>
        </pagedown-editor>
        <button type="button" class="btn btn-primary" ng-click="submitAnswerComment(answer)" ng-disabled="answer_{{answer.id}}_comment.$invalid">Add Comment</button>
      </form>
    </div>
  </div>
  <hr/>
  <h3>Your answer</h3>
  ...

client/app/questionsShow/questionsShow.scss

.comment-wmd-input {
  height: 50px;
  width: 100%;
  background-color: Gainsboro;
  border: 1px solid DarkGray;
};

Adding stars

Adding a feature to star or unstar questions, answers, and comments on questions or answers.

Editing server-side database model

Store the list of starring users for a question, an answer, and a comment on a question or answer to a "stars" field of a question document as an array.

server/api/question/question.model.js

var QuestionSchema = new mongoose.Schema({
  ...
  answers: [{
    ...
    comments: [{
      ...
      stars: [{
        type: mongoose.Schema.ObjectId,
        ref: 'User'
      }],
      ...
    }],
    stars: [{
      type: mongoose.Schema.ObjectId,
      ref: 'User'
    }],
    ...
  }],
  ...
  comments: [{
    ...
    stars: [{
      type: mongoose.Schema.ObjectId,
      ref: 'User'
    }],
    ...
  }],
  stars: [{
    type: mongoose.Schema.ObjectId,
    ref: 'User'
  }],
  ...
});

Adding server-side routing

Add the following APIs to star or unstar a question, an answer, or a comment on a question or an answer.

Method URL Description
POST /:id/star Star a question
DELETE /:id/star Unstar a question
POST /:id/answers/:answerId/star Star an answer
DELETE /:id/answers/:answerId/unstar Unstar an answer
POST /:id/comments/:commentId/star Star a comment on a question
DELETE /:id/comments/:commentId/unstar Unstar a comment on a question
POST /:id/answers/:answerId/comments/:commentId/star Star a comment on an answer
DELETE /:id/answers/:answerId/comments/:commentId/star Unstar a comment on an answer

Because these APIs are related to user, register "auth.isAuthenticate()" to enable authentication.

server/api/question/index.js

router.put('/:id/star', auth.isAuthenticated(), controller.star);
router.delete('/:id/star', auth.isAuthenticated(), controller.unstar);
router.put('/:id/answers/:answerId/star', auth.isAuthenticated(), controller.starAnswer);
router.delete('/:id/answers/:answerId/star', auth.isAuthenticated(), controller.unstarAnswer);
router.put('/:id/comments/:commentId/star', auth.isAuthenticated(), controller.starComment);
router.delete('/:id/comments/:commentId/star', auth.isAuthenticated(), controller.unstarComment);
router.put('/:id/answers/:answerId/comments/:commentId/star', auth.isAuthenticated(), controller.starAnswerComment);
router.delete('/:id/answers/:answerId/comments/:commentId/star', auth.isAuthenticated(), controller.unstarAnswerComment);

Editing server-side database model

On the database model, for questions, answers, and comments on questions or answers, store the list of starred users as an array. On "update()" function, we can refer the matched index of an array using "$". For the list of a starred users of a comment array of an answer array, because we can only use one "$", we need to iterate the array explicitly.

server/api/question/question.controller.js

/* star/unstar question */
export function star(req, res) {
  Question.update({_id: req.params.id}, {$push: {stars: req.user.id}}, function(err, num){
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
}
export function unstar(req, res) {
  Question.update({_id: req.params.id}, {$pull: {stars: req.user.id}}, function(err, num){
    if(err) { return handleError(res, err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
}

/* star/unstar answer */
export function starAnswer(req, res) {
  Question.update({_id: req.params.id, 'answers._id': req.params.answerId}, {$push: {'answers.$.stars': req.user.id}}, function(err, num){
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
}
export function unstarAnswer(req, res) {
  Question.update({_id: req.params.id, 'answers._id': req.params.answerId}, {$pull: {'answers.$.stars': req.user.id}}, function(err, num){
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
}

/* star/unstar question comment */
export function starComment(req, res) {
  Question.update({_id: req.params.id, 'comments._id': req.params.commentId}, {$push: {'comments.$.stars': req.user.id}}, function(err, num){
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
}
export function unstarComment(req, res) {
  Question.update({_id: req.params.id, 'comments._id': req.params.commentId}, {$pull: {'comments.$.stars': req.user.id}}, function(err, num){
    if(err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
}

/* star/unstar question answer comment */
var pushOrPullStarAnswerComment = function(op, req, res) {
  Question.find({_id: req.params.id}).exec(function(err, questions){
    if(err) { return handleError(res)(err); }
    if(questions.length === 0) { return res.send(404).end(); }
    var question = questions[0];
    var found = false;
    for(var i=0; i < question.answers.length; i++){
      if(question.answers[i]._id.toString() === req.params.answerId){
        found = true;
        var conditions = {};
        conditions._id = req.params.id;
        conditions['answers.' + i + '.comments._id'] = req.params.commentId;
        var doc = {};
        doc[op] = {};
        doc[op]['answers.' + i + '.comments.$.stars'] = req.user.id;
        // Question.update({_id: req.params.id, 'answers.' + i + '.comments._id': req.params.commentId}, {op: {('answers.' + i + '.comments.$.stars'): req.user.id}}, function(err, num){
        /*jshint -W083 */
        Question.update(conditions, doc, function(err, num){
          if(err) { return handleError(res)(err); }
          if(num === 0) { return res.send(404).end(); }
          exports.show(req, res);
          return;
        });
      }
    }
    if(!found){
      return res.send(404).end();
    }
  });
};
export function starAnswerComment(req, res) {
  pushOrPullStarAnswerComment('$push', req, res);
}
export function unstarAnswerComment(req, res) {
  pushOrPullStarAnswerComment('$pull', req, res);
}

Editing client-side question-showing controller

Add functions to star, unstar, or check staring status for a question, an answer, or a comment on a question or an answer. We use a sub-pathname parameter to use the common function for questions, answers, or comments.

client/app/questionsShow/questionsShow.controller.js

    $scope.isStar = function(obj){
      return Auth.isLoggedIn() && obj && obj.stars && obj.stars.indexOf(Auth.getCurrentUser()._id)!==-1;
    };
    $scope.star = function(subpath) {
      $http.put('/api/questions/' + $scope.question._id + subpath + '/star').success(function(){
        loadQuestions();
      });
    };
    $scope.unstar = function(subpath) {
      $http.delete('/api/questions/' + $scope.question._id + subpath + '/star').success(function(){
        loadQuestions();
      });
    };

Editing client-side question-showing HTML file

Show the staring status as the star icon. We can click the star icon to star or unstar.

client/app/questionsShow/questionsShow.html

    <div style="float: left;font-size: x-large; padding: 0; width: 2em; text-align: center;">
      <button ng-if=" isStar(question)" type="button" style="background: transparent; border: 0;" ng-click="unstar('')">
        <span class="glyphicon glyphicon-star" style="color: #CF7C00;" ></span>
      </button>
      <button ng-if="!isStar(question)" type="button" style="background: transparent; border: 0;" ng-click="star('')"  >
        <span class="glyphicon glyphicon-star-empty"></span>
      </button>
      <br/>
      <div>{{question.stars.length}}</div>
    </div>
    
    <div>
      <h1>
        <div ng-if="! editing">{{question.title}}</div>

client/app/questionsShow/questionsShow.html

      <div style="float: left;font-size: normal; padding: 0; width: 2em; text-align: center;">
        <button ng-if=" isStar(comment)" type="button" style="background: transparent; border: 0;" ng-click="unstar('/comments/' + comment._id)">
          <span class="glyphicon glyphicon-star" style="color: #CF7C00;" ></span>
        </button>
        <button ng-if="!isStar(comment)" type="button" style="background: transparent; border: 0;" ng-click="  star('/comments/' + comment._id)"  >
          <span class="glyphicon glyphicon-star-empty"></span>
        </button>
        <br/>
        <div>{{comment.stars.length}}</div>
      </div>

      <pagedown-viewer content="comment.content" ng-if="!editing"></pagedown-viewer>

client/app/questionsShow/questionsShow.html

    <div style="float: left;font-size: large; padding: 0; width: 2em; text-align: center;">
      <button ng-if=" isStar(answer)" type="button" style="background: transparent; border: 0;" ng-click="unstar('/answers/' + answer._id)">
        <span class="glyphicon glyphicon-star" style="color: #CF7C00;" ></span>
      </button>
      <button ng-if="!isStar(answer)" type="button" style="background: transparent; border: 0;" ng-click="  star('/answers/' + answer._id)"  >
        <span class="glyphicon glyphicon-star-empty"></span>
      </button>
      <br/>
      <div>{{answer.stars.length}}</div>
    </div>

    <div class="answer">
        <div style="float: left;font-size: normal; padding: 0; width: 2em; text-align: center;">
          <button ng-if=" isStar(comment)" type="button" style="background: transparent; border: 0;" ng-click="unstar('/answers/' + answer._id + '/comments/' + comment._id)">
            <span class="glyphicon glyphicon-star" style="color: #CF7C00;" ></span>
          </button>
          <button ng-if="!isStar(comment)" type="button" style="background: transparent; border: 0;" ng-click="  star('/answers/' + answer._id + '/comments/' + comment._id)"  >
            <span class="glyphicon glyphicon-star-empty"></span>
          </button>
          <br/>
          <div>{{comment.stars.length}}</div>
        </div>

        <pagedown-viewer content="comment.content" ng-if="!editing"></pagedown-viewer>

Editing client-side question-listing controller

Add "isStar()" function to show the staring status on question listing.

client/app/questionsIndex/questionsIndex.controller.js

  .controller('QuestionsIndexCtrl', function ($scope, $http, Auth, $location) {
    ...
    $scope.isStar = function(obj){
      return Auth.isLoggedIn() && obj && obj.stars && obj.stars.indexOf(Auth.getCurrentUser()._id)!==-1;
    };

Editing client-side question-listing HTML file

On question listing, show the number of staring users for the question and the number of answers.

client/app/questionsIndex/questionsIndex.html

  <table class="table table-striped">
    <thead>
      <tr>
        <th width="20">Stars</th>
        <th width="20">Answers</th>
        <th>Question</th>
      </tr>
    </thead>
    <tbody>
      <tr ng-repeat="question in questions">
        <td style="text-align: center; vertical-align:middle">
          <div style="font-size: xx-large;">{{question.stars.length}}</div>
        </td>
        <td style="text-align: center; vertical-align:middle">
          <div style="font-size: xx-large;">{{question.answers.length}}</div>
        </td>
        <td>
          <div style="float: right;">
            <span ng-if=" isStar(question)" class="glyphicon glyphicon-star" style="color: #CF7C00;" ></span>
            <span ng-if="!isStar(question)" class="glyphicon glyphicon-star-empty"></span>
          </div>
          <a ng-href="/questions/show/{{question._id}}" style="font-size: large">{{question.title}}</a>
          <div class="clearfix"></div>
          <div style="float: right;">
            by <a ng-href="/users/{{question.user._id}}">{{question.user.name}}</a>
             · {{question.createdAt|fromNow}}
          </div>
          <div>
            <span ng-repeat="tag in question.tags">
              <span class="label label-info">
                {{tag.text}}
              </span>
              &nbsp;
            </span>
          </div>
          <div class="clearfix"></div>
        </td>
      </tr>
    </tbody>
  </table>

Question listing for all questions, my questions, and starred questions

For now, all questions are always listed. Let's enable to choose from all questions, my questions, and starred questions.

Editing client-side routing

Assign the following URLs for all questions, my questions, and starred questions.

URL Description
/ All questions
/users/:userId My questions
/users/:userId/starred Starred questions

Use the same controller and HTML template and change the searching query by "query" variable. For my questions listing, the query checks whether the user of the question is the same as the current login user. For starred questions listing, the query checks whether the starred user for the question, the answers, or the comments contains current login user.

client/app/questionsIndex/questionsIndex.js

...
  .config(function ($stateProvider) {
    $stateProvider
      .state('main', {
        url: '/',
        templateUrl: 'app/questionsIndex/questionsIndex.html',
        controller: 'QuestionsIndexCtrl',
        resolve: {
          query: function(){return {};}
        },
      })
      .state('starredQuestionsIndex', {
        url: '/users/:userId/starred',
        templateUrl: 'app/questionsIndex/questionsIndex.html',
        controller: 'QuestionsIndexCtrl',
        resolve: {
          query: function($stateParams){
            return {
              $or: [
                {'stars': $stateParams.userId},
                {'answers.stars': $stateParams.userId},
                {'comments.stars': $stateParams.userId},
                {'answers.comments.stars': $stateParams.userId},
              ]
            };
          }
        },
      })
      .state('userQuestionsIndex', {
        url: '/users/:userId',
        templateUrl: 'app/questionsIndex/questionsIndex.html',
        controller: 'QuestionsIndexCtrl',
        resolve: {
          query: function($stateParams){
            return {user: $stateParams.userId};
          }
        },
      });

  });

Editing client-side question-listing controller

Send the query set on routing to the server.

client/app/questionsIndex/questionsIndex.controller.js

  .controller('QuestionsIndexCtrl', function ($scope, $http, Auth, query) {
    $http.get('/api/questions', {params: {query: query}}).success(function(questions) {

Editing client-side question-listing controller test

Add empty a "query" to test code not to cause an error.

client/app/questionsIndex/questionsIndex.controller.spec.js

    QuestionsIndexCtrl = $controller('QuestionsIndexCtrl', {
      $scope: scope,
      query: {},
    });

Editing server-side controller

Send the received query to the database.

server/api/question/question.controller.js

export function index(req, res) {
  var query = req.query.query && JSON.parse(req.query.query);
  Question.find(query).sort(...

Editing client-side Navbar controller

On Navbar, add links for all questions, my questions, and ffffffstarred questions. Because we need to change the URL or enable/disable for links before login or logout, use functions instead of variables for that information on menu items.

client/components/navbar/navbar.controller.js

  constructor(Auth) {
    this.menu = [
      {
        'title': 'All',
        'link': function(){return '/';},
        'show': function(){return true;},
      },
      {
        'title': 'Mine',
        'link': function(){return '/users/' + Auth.getCurrentUser()._id;},
        'show': Auth.isLoggedIn,
      },
      {
        'title': 'Starred',
        'link': function(){return '/users/' + Auth.getCurrentUser()._id + '/starred';},
        'show': Auth.isLoggedIn,
      },
    ];
    this.isLoggedIn = Auth.isLoggedIn;
    this.isAdmin = Auth.isAdmin;
    this.getCurrentUser = Auth.getCurrentUser;
  }
  ...

Editing client-side Navbar HTML

Retrieve linked URLs using "$scope.item.link()" function. Also, show the link only when "$scope.item.show()" returns true.

client/components/navbar/navbar.html

      <ul class="nav navbar-nav">
        <li ng-repeat="item in nav.menu" ng-class="{active: isActive(item.link())}" ng-show="item.show()">
            <a ng-href="{{item.link()}}">{{item.title}}</a>
        </li>
      </ul>

Search

With MongoDB's full-text search, let's add a feature to search on question titles, contents, comments, or answers.

Editing client-side Navbar HTML file

Add serach box to Navbar. Call "search()" function on search.

client/components/navbar/navbar.html:

      <form class="navbar-form navbar-left" role="search" ng-submit="nav.search(keyword)">
        <div class="input-group">
          <input type="text" class="form-control" placeholder="Search" ng-model="keyword">
          <span class="input-group-btn">
            <button type="submit" class="btn btn-default">
              <span class="glyphicon glyphicon-search">
              </span>
            </button>
          </span>
        </div>
      </form>
      <ul class="nav navbar-nav navbar-right">

Editing client-side Navbar controller

client/components/navbar/navbar.controller.js

  constructor(Auth, $state) {
    this.search = function(keyword) {
      $state.go('main', {keyword: keyword}, {reload: true});
    };
    ...

Editing server-side database model

To add index for a full-text search for question titles, question contents, comments, and answers, use "QuestionSchema.index()" function and specify search target fields as 'text'. Although MongoDB can automatically set index name, because MongoDB does not work as intended if the length of the name is long and exceeds 128 bytes, let's explicitly specify the index name (ex: 'question_schema_index'). (Ref: We need to specify name explicitly for long schema.)

server/api/question/question.model.js

QuestionSchema.index({
  'title': 'text',
  'content': 'text',
  'tags.text': 'text',
  'answers.content': 'text',
  'comments.content': 'text',
  'answers.comments.content': 'text',
}, {name: 'question_schema_index'});

Editing client-side question-listing routing

To accept search keyword as "keyword" URL parameter, add "/?keyword" to the "url" field of the routing information.

client/app/questionsIndex/questionsIndex.js

      .state('main', {
        url: '/?keyword',
        ...

Editing client-side question-showing controller

Set the query using MongoDB's '$text' and '$search' parameters to search by 'keyword' parameter.

client/app/questionsIndex/questionsIndex.controller.js

  .controller('QuestionsIndexCtrl', function ($scope, $http, $location, query) {
    ...
    var keyword = $location.search().keyword;
    if(keyword){
      query = _.merge(query, {$text: {$search: keyword}});
    }
    $http.get('/api/questions',...

Japanese search

MongoDB's full-text search only supports Latin languages, and does not support other languages such as Japanese. Let's support Japanese search by using Japanese tokenizer "TinySegmenter".

Installing TinySegmenter library

Install TinySegmenter as a server-side library.

% npm install --save r7kamura/tiny-segmenter

On the server-side DB model, add "searchText" field to store tokenized text.

server/api/question/question.model.js

var QuestionSchema = new mongoose.Schema({
  ...
  searchText: String,
});

Add "searchText" field to the text index.

server/api/question/question.model.js

QuestionSchema.index({
  ...
  'searchText': 'text',
}, {name: 'question_schema_index'});

Add a function to tokenize, and call it before saving by using "pre('save')" hook. Also, add a static function to the question model by adding the function to "QuestionSchema.statics".

server/api/question/question.model.js

var TinySegmenter = require('tiny-segmenter');

var getSearchText = function(question){
  var tinySegmenter = new TinySegmenter();
  var searchText = "";
  searchText += tinySegmenter.segment(question.title).join(' ') + " ";
  searchText += tinySegmenter.segment(question.content).join(' ') + " ";
  question.answers.forEach(function(answer){
    searchText += tinySegmenter.segment(answer.content).join(' ') + " ";
    answer.comments.forEach(function(comment){
      searchText += tinySegmenter.segment(comment.content).join(' ') + " ";
    });
  });
  question.comments.forEach(function(comment){
    searchText += tinySegmenter.segment(comment.content).join(' ') + " ";
  });
  console.log("searchText", searchText);
  return searchText;
};
QuestionSchema.statics.updateSearchText = function(id, cb){
  this.findOne({_id: id}).exec(function(err, question){
    if(err){ if(cb){cb(err);} return; }
    var searchText = getSearchText(question);
    this.update({_id: id}, {searchText: searchText}, function(err, num){
      if(cb){cb(err);}
    });
  }.bind(this));
};

QuestionSchema.pre('save', function(next){
  this.searchText = getSearchText(this);
  next();
});

Because "pre('save')" hook is not called by "update()" call, explicitly call "updateSearchText()" function to tokenize and update the full-text index.

server/api/question/question.controller.js

export function createAnswer(req, res) {
    ...
    exports.show(req, res);
    Question.updateSearchText(req.params.id);
  });
};
export function destroyAnswer(req, res) {
  ...
    exports.show(req, res);
    Question.updateSearchText(req.params.id);
  });
};
export function updateAnswer(req, res) {
    ...
    exports.show(req, res);
    Question.updateSearchText(req.params.id);
  });
};
...
/* comments APIs */
export function createComment(req, res) {
    ...
    exports.show(req, res);
    Question.updateSearchText(req.params.id);
  })
};
export function destroyComment(req, res) {
    ...
    exports.show(req, res);
    Question.updateSearchText(req.params.id);
  });
};
export function updateComment(req, res) {
    ...
    exports.show(req, res);
    Question.updateSearchText(req.params.id);
  });
};
...
/* answersComments APIs */
export function createAnswerComment(req, res) {
    ...
    exports.show(req, res);
    Question.updateSearchText(req.params.id);
  })
};
export function destroyAnswerComment(req, res) {
    ...
    exports.show(req, res);
    Question.updateSearchText(req.params.id);
  });
};
export function updateAnswerComment(req, res) {
  ...
          exports.show(req, res);
          Question.updateSearchText(req.params.id);
  ...
};

To re-create index, drop the question collection and restart the server(grunt serve).

% mongo
> use paizaqa-dev
> db.questions.drop()
% grunt serve

Infinite scroll

For now, we can list only the last 20 questions. Let's add an infinite scroll to show older questions.

Installing ngInfiniteScroll library

Install ngInfiniteScroll module for infinite scroll as a client-side library.

% bower install --save ngInfiniteScroll

Adding ngInfiniteScroll to the application modules

Add ngInfiniteScroll to the application depending modules to use.

client/app/app.js

angular.module('paizaqaApp', [
  ...
  'infinite-scroll',
 ])

Editing client-side question-listing controller

To list older questions on a scroll, implement "nextPage()" function. To retrieve questions older than the current oldest question ID, specify the query as "{_id: {$lt: lastID}}". Store the loading status as "$scope.busy", and the status of whether more data is available as "$scope.noMoreData".

client/app/questionsIndex/questionsIndex.controller.js

    $scope.busy = true;
    $scope.noMoreData = false;

    $http.get('/api/questions', {params: {query: query}}).success(function(questions) {
      $scope.questions = questions;
      if($scope.questions.length < 20){
        $scope.noMoreData = true;
      }
      $scope.busy = false;
    });
    ...
    $scope.nextPage = function(){
      if($scope.busy){ return; }
      $scope.busy = true;
      var lastId = $scope.questions[$scope.questions.length-1]._id;
      var pageQuery = _.merge(query, {_id: {$lt: lastId}});
      $http.get('/api/questions', {params: {query: pageQuery}}).success(function(questions){
        $scope.questions = $scope.questions.concat(questions);
        $scope.busy = false;
        if(questions.length === 0){
          $scope.noMoreData = true;
        }
      });
    };

Editing client-side question listing

To load older questions on scroll to the bottom, add "infinite-scroll" attribute to call "nextPage()" function. Disable scroll when loading or when there is no more data. Show "Loading data" on loading by adding an element with "ng-show='busy'", so the element is shown only when the "busy" variable is "true".

client/app/questionsIndex/questionsIndex.html

<div class="container" infinite-scroll='nextPage()' infinite-scroll-disabled='busy || noMoreData'>
  ...
  <div ng-show='busy'>Loading data...</div>
</div>

SNS authentication

When using SNS authentication(Facebook, Twitter, Google), set API key and SECRET key. See an instruction in the first article for details.

For Facebook authentication, because API specification changed on Graph API 2.4 (on July 9th, 2015), we need to explicitly specify fields to use on "profileFields" as follows.

Deploying

Now, let's deploy the application to Heroku. To configure Heroku deployment, use "yo angular-fullstack:heroku" command. Also, install a MongoDB module MongoLab to the Heroku application, as MongoLab has a free plan.

% yo angular-fullstack:heroku
% cd dist
% heroku addons:add mongolab

From the next deployment, use "grunt" to build the application, and use "grunt buildcontrol:heroku" to deploy the application.

% grunt
% grunt buildcontrol:heroku

The application is deployed. Open http://APPLICATION.herokuapp.com/ to see the application !

When it does not work, see Heroku logs.

% cd dist
% heroku logs

For browsing MongoDB, GUI tools such as MongoHub are convenient. We can retrieve the URL for the MongoDB database from Heroku using "heroku config" command.

% heroku config
...
MONGOLAB_URI: mongodb://USERNAME:PASSWORD@HOSTNAME:PORT/DATABASE
...

Summary

I introduced how to build a Q&A service using a MEAN stack, AngularJS Full-Stack generator. Using MEAN stack, we can quickly build a sophisticated web service, from client-side logic to server-side logic, by only using JavaScript. Using the generator, we can get best practice boilerplate codes, and just edit the codes to create services. It is especially helpful for startups or prototypes where it is necessary to create and change services quickly by trial and error.

Lets' come up with ideas, and build your own services!

I welcome any feedback such as errors, suggestion, or anything you noticed about this articles as comments!

I'll continue writing articles to build web services using MEAN stack.


MEAN stack development articles
Building full-stack web service - MEAN stack development(1)
Building Twitter-like full-stack web service in 1 hour - MEAN stack development (2)
* Building a Q&A web service in an hour - MEAN stack development(3)

Building Twitter-like full-stack web service in 1 hour - MEAN stack development (2)

(Japanese article is here.)

f:id:paiza:20140712194904j:plain (by Yoshioka Tsuneo, @ at https://paiza.IO/)

In the previous article, I introduced a MEAN stack, Yeoman-based AngularJS Full-Stack generator, and explained how to install, run, edit, debug, and deploy programs. MEAN stack is a web development package of MongoDB, Express, AngularJS, and Node.js. You can easily build interactive and intuitive full-stack web services by just using one language: JavaScript.

In this article, we'll build a more practical real web service!

The web service is a Twitter-like service where users can post and list messages. We can build a full-fledged, nearly production-ready web service based by editing some JavaScript or HTML code generated. Let's try!

f:id:paiza:20150706130351p:plain

Demo: http://paizatter.herokuapp.com

The web service has features like below:

  • Sign Up, and Login
  • Post, remove, or list messages
  • Search posted messages
  • Infinite scrolling list
  • Starred messages (Add, remove, or list)

You can download the source code below. But, I suggest writing code by your hands to have a better understanding for the codes.

https://github.com/gi-no/paizatter

Contents

Install MEAN stack

Install Yeoman-based AngularJS Full-Stack generator(generator-angular-fullstack) following the instructions in the previous article.

Confirm that installed AngularJS Full-Stack generator is ver3.0.0 or later.

$ npm ls -g generator-angular-fullstack
/usr/local/lib
└── generator-angular-fullstack@3.0.0-rc4 

If it is older than ver3.0.0, update to the latest version.

$ sudo npm update -g generator-angular-fullstack

Create a new project

First, let's create a new project. Create a project directory and run "yo"(Yeoman) command. I named the project "paizatter".

$ mkdir paizatter
$ cd paizatter
$ yo angular-fullstack paizatter

There are multiple configurations from which to choose. Let's just choose default settings except for the SNS setting where we enable all the SNS.

- Would you like to include additional oAuth strategies? 
 ◉ Google
 ◉ Facebook
❯◉ Twitter

After a minute, many project files are generated. The following are some of project files related to this article.

.
|-- bower.json                            Bower packages(Client-side libraries)
|-- package.json                          npm packages(Server-side libraries)
|
|-- client                                Client-side codes
|   |-- app
|   |   |-- app.js                        Client-side main JavaScript code
|   |   `-- main
|   |       |-- main.controller.js        Client-side controller code
|   |       |-- main.controller.spec.js   Client-side test code
|   |       |-- main.html                 HTML template file
|   |       |-- main.js                   Client-side routing configuration
|   |       `-- main.scss                 CSS file
|   |-- components
|   |   |-- navbar
|   |   |   |-- navbar.controller.js      Navbar controller
|   |   |   `-- navbar.html               Navbar HTML template file
|   |   `-- socket
|   |       `-- socket.service.js         Client-side WebSocket code
|   `-- index.html
|
`-- server                                Server-side code
    `-- api
        `-- thing
            |-- index.js                  Server-side API routing configuration
            |-- thing.controller.js       Server-side controller(API implementation)
            |-- thing.model.js            Server-side DB model
            |-- thing.socket.js           Server-side WebSocket implementation
            `-- thing.integration.js      Server-side test code

Client-side codes are under the "client" directory, and server-side codes are under the "server" directory.

In the "client/app" directory, each page has its own directory (ex: "client/app/main"). The directory pack a JavaScript code (controller), a HTML file (view), a URL routing configuration file, a CSS files, and a test file to make it easy to maintain.

In the "server/api" directory, each subdirectory has own JavaScript API code(controller), Web socket code, URL routing configuration, test code, and DB model.

f:id:paiza:20150709030234p:plain

Client-side controllers communicate with server-side controllers using the server APIs, and they update HTML page or handle events. Server-side controllers communicate with client-side controllers using server APIs, and they retrieve or update data from/to MongoDB through the DB model.

If we think about MVC, from the clients' point of view, servers are like models. From servers' point of view, clients are like views.

The default npm packages on Angular Full-stack generator are a bit old, so update to the latest packages using "npm-check-updates" .

% sudo npm install -g npm-check-updates
% npm-check-updates -u
% npm install

Now, start the server.

% grunt serve

List messages

Generated project have a controller(client/app/main/main.controller.js) that owns "things" array object. The main page list the "things" object.

In this project, we use the "thing" object to store messages.

Open the HTML file and edit a "div" element with "container" class.

client/app/main/main.html:

<div class="container">
  <br/>
  <form>
    <div class="input-group">
      <input type="text" class="form-control" placeholder="Message" ng-model="newThing">
      <span class="input-group-btn">
        <button type="submit" class="btn btn-primary" ng-click="addThing()">Add New</button>
      </span>
    </div>
  </form>

  <div class="row">
    <div ng-repeat="thing in awesomeThings">
      {{thing.name}}
    </div>
  </div>
</div>

Now, the main page shows the input form and message list.

You see "ng-repeat" or "{{expression}}" that is not used in HTML. Those are AngularJS syntax to embed JavaScript variables on the HTML templates.

"ng-repeat" is an AngularJS syntax to expand an array object in HTML templates. You can write "ng-repeat=ITEM in ARRAY" to output each object in the array. You can use "{{expression}}" syntax to embed a variable or simple expression(Angular Expression) in the HTML templates.

f:id:paiza:20150706134637p:plain

Change order of the list

Now, we see the message list, but new messages are appended on the bottom of the list, instead of on the top of the list. Let's append a new message on the top of the list like Twitter does. Also, let's show only the last 20 messages instead of all the messages.

On WebSocket code, use "push" instead of "unshift" to append a new item on the top of the messages.

client/components/socket/socket.service.js:

      syncUpdates: ...
        socket.on(...
          ...
          // array.push(item);
          array.unshift(item);

Callback code inside "socket.on()" is called using WebSocket when the message list is updated. Thanks to WebSocket, we can automatically update the message list without manually reloading page when other users add a message,

To change the order on loading messages when initially loading the page or reloading the page, edit the server-side controller for the listing message.

server/api/thing/thing.controller.js:

// Gets a list of Things
exports.index = function(req, res) {
  Thing.find().sort({_id:-1}).limit(20).execAsync()
    .then(responseWithResult(res))
    .catch(handleError(res));
};

Use sort() function of mongoose (MongoDB middleware), to sort in descending order by creation time. MongoDB has "id" field on every document (RDB's record), and the "id" field is ordered by creation time. So, we can just sort by "_id" field to sort by creation time.

limit() function limits the number of object to return. When the query is built, call exec() function to call query. The query result is returned as an argument of the callback function. So, just return the result to the client.

User authentication

Now, we can list the messages, but we don't see who posted the messages. Also, only posted user should be able to delete his/her message.

So, let's add user authentication.

Sign Up and Login feature is already generated from the template, so we just need to add authentication for features and show the user name on the messages.

Server-side model schema

Store the user ID and the message together. MongoDB itself has flexible schema, but AngularJS Full-Stack generator also uses mongoose as a driver. Mongoose has features such as removing needless fields on save, hook functions, and expanding related documents.

On the mongoose schema configuration for messages(ThingSchema), add user ID to message schema. The "name" field stores a message, "user" field stores User's ObjectID. "ref: 'User'" relates the ObjectID to the User collection, and enables to expand using populate() function.

Also, add creation time. The "createAt" field have "Date.now()" function as a default value to set creation time automatically.

server/api/thing/thing.model.js:

var ThingSchema = new Schema({
  name: String, /* message */
  user: {
    type: Schema.ObjectId,
    ref: 'User'
  },
  createdAt: {
      type: Date,
      default: Date.now
  },
});

And, find() or findOne() query just returns the ObjectID of User instead of the User object itself. Use the populate() function to expand User object. populate('user') expand all fields of the User object. Specify populate('user','name') to just expand a needed field('name').

Although we can expand on each query, to expand for all query, use "pre()" to hook all 'find()', 'findOne()' call and call populate().

ThingSchema.pre('find', function(next){
  this.populate('user', 'name');
  next();
});
ThingSchema.pre('findOne', function(next){
  this.populate('user', 'name');
  next();
});

Server-side API routing configuration

For APIs requiring authentication, add "auth.isAuthenticated()" middleware to the routings. In this way, posting or deleting messages from unauthorized users is prohibited, and request object ("req") have user field ("req.user") to store User object.

server/api/thing/index.js:

var auth = require('../../auth/auth.service');

router.get('/', controller.index);
router.get('/:id', controller.show);
router.post('/', auth.isAuthenticated(), controller.create);
router.delete('/:id', auth.isAuthenticated(), controller.destroy);

On the routing file above, we can specify a function called when each URL is requested. To add authentication, specify auth.isAuthenticated() as a middleware. Also, remove needless put/patch routing.

Edit server-side controller(create function)

On the server-side controller, add the user object to "user" field of creating document (req.body.user = req.user). Because "req.user" already contains the user object, just set it to Thing.create argument ("req.body") to save user.

server/api/thing/thing.controller.js:

// Creates a new Thing in the DB
exports.create = function(req, res) {
  req.body.user = req.user;
  Thing.createAsync(req.body)
  ...

Edit server-side controller to delete messages.

On deletion, validate that the posting user and the current user are the same before deletion.

server/api/thing/thing.controller.js:

function handleUnauthorized(req, res) {
  return function(entity) {
    if (!entity) {return null;}
    if(entity.user._id.toString() !== req.user._id.toString()){
      res.send(403).end();
      return null;
    }
    return entity;
  }
}
...
// Deletes a Thing from the DB
exports.destroy = function(req, res) {
  Thing.findByIdAsync(req.params.id)
    .then(handleEntityNotFound(res))
    .then(handleUnauthorized(req, res))
    .then(removeEntity(res))
    .catch(handleError(res));
};

Edit client-side controller

Add isMyTweet() function to check whether the message is of current user or not.

client/app/main/main.controller.js:

angular.module('paizatterApp')
  .controller('MainCtrl', function ($scope, $http, socket, Auth) {
    $scope.isLoggedIn = Auth.isLoggedIn;
    $scope.getCurrentUser = Auth.getCurrentUser;
...
    $scope.isMyTweet = function(thing){
      return Auth.isLoggedIn() && thing.user && thing.user._id===Auth.getCurrentUser()._id;
    };
  });

On the above controller function, arguments of the controller function specify modules to use. So, add "Auth" to the controller function arguments.

Variables or functions stored in $scope object can be referred on HTML code, so add a function as "$scope.isMyTweet". "isMyTweet" function checks whether the message's user ID is the same as the current user ID or not. Also, to call authentication function from HTML templates, add isLoggedIn/getCurrentUser to $scope object.

Edit client-side HTML template

On the message listing, add the username and creation time.

client/app/main/main.html:

  <div ng-repeat="thing in awesomeThings">
    <div class="row">
      {{thing.user.name}} - {{thing.name}} ({{thing.createdAt}})
      <button ng-if="isMyTweet(thing)" type="button" class="close" ng-click="deleteThing(thing)">&times;</button>
    </div>
  </div>

Edit server-side test

In this project, remove routing test.

% rm server/api/thing/index.spec.js

For APIs requiring authentication, before each test, login and set authentication information before the test. Also, remove a test for "PUT API" which we don't use.

server/api/thing/thing.integration.js:

var User = require('../user/user.model');
...
describe('Thing API:', function() {
  var user;
  before(function() {
    return User.removeAsync().then(function() {
      user = new User({
        name: 'Fake User',
        email: 'test@test.com',
        password: 'password'
      });

      return user.saveAsync();
    });
  });

  var token;
  before(function(done) {
    request(app)
      .post('/auth/local')
      .send({
        email: 'test@test.com',
        password: 'password'
      })
      .expect(200)
      .expect('Content-Type', /json/)
      .end(function(err, res) {
        token = res.body.token;
        done();
      });
  });
  ...    
  describe('POST /api/things', function() {
    ...    
        .post('/api/things')
        .set('authorization', 'Bearer ' + token)
    ...
  describe('DELETE /api/things/:id', function() {
    ...
        .delete('/api/things/' + newThing._id)
        .set('authorization', 'Bearer ' + token)
    ...
        .delete('/api/things/' + newThing._id)
        .set('authorization', 'Bearer ' + token)
    ...
  /* describe('PUT /api/things/:id', function() {
  }); */

Test

Now, all authentication feature have been implemented, so let's test it.

If you post without Login, you are redirected to Sign Up page. The posted message contains the username. You can only delete your message (using the cross("x") button).

f:id:paiza:20150706135436p:plain

Edit CSS

The current message list has no decoration. Add CSS to decorate message.

CSSARROW

Choose an arrow CSS from http://cssarrowplease.com . Just choose your favorite style and append it to "main.scss".

client/app/main/main.scss:

// http://cssarrowplease.com
.arrow_box {
    position: relative;
    background: #f0f0f0;
    border: 4px solid #c2e1f5;
}
.arrow_box:after, .arrow_box:before {
    right: 100%;
    top: 50%;
    border: solid transparent;
    content: " ";
    height: 0;
    width: 0;
    position: absolute;
    pointer-events: none;
}

.arrow_box:after {
    border-color: rgba(224, 224, 224, 0);
    border-right-color: #f0f0f0;
    border-width: 10px;
    margin-top: -10px;
}
.arrow_box:before {
    border-color: rgba(194, 225, 245, 0);
    border-right-color: #c2e1f5;
    border-width: 16px;
    margin-top: -16px;
}

Also, add margin or set font.

client/app/main/main.scss:

.tweet{
    margin: 5px;
}
.arrow_box .message {
    font-size: 16px;
    height: 2em;
}

Edit the HTML file

Edit the HTML file to apply CSS styles.

client/app/main/main.html:

  <div ng-repeat="thing in awesomeThings" class="tweet">
    <div class="row">
      <h2 class="col-xs-2">
        {{thing.user.name}}
      </h2>
      <div class="arrow_box col-xs-10">
        <button ng-if="isMyTweet(thing)" type="button" class="close" ng-click="deleteThing(thing)">&times;</button>
        <h2 class="message">
          {{thing.name}} 
        </h2>
        <span style="float: right;">({{thing.createdAt}})</span>
      </div>
    </div>
  </div>

f:id:paiza:20150706140631p:plain

Deploy

At this point, basic features are done!

Let's deploy for now.

% yo angular-fullstack:heroku
% cd dist
% heroku addons:add mongolab

The "yo angular-fullstack:heroku" command will set up a deploy environment for Heroku. Also, add MongoDB module to Heroku. Heroku provides MongoHQ and MongoLab as MongoDB add-ons. Let's add MongoLab add-on because MongoLab has a free plan.

Now, we deployed the web service to Heroku ! For next deployments, use the "grunt" command to build a distribution package, and "grunt buildcontrol:heroku" for deployment.

% grunt
% grunt buildcontrol:heroku

Now, it's time to open a browser to use the web service!

http://APPLICATION-NAME.herokuapp.com/

SNS authentication

To use SNS authentication(Facebook, Twitter, Google), set up API key and SECRET key. Please refer to the instructions in the previous article.

Debug

In case you failed deployment, check out the server log file. Please refer to the instruction in the previous article for details.

% cd dist
% heroku logs

For about MongoDB operation, GUI tools like MongoHub are helpful. MongoDB URL can be retrieved from the Heroku configuration.

% heroku config ... MONGOLAB_URI: mongodb://Username:Password@Hostname:Port/Database ...

Create time format filter

Now, the message creation times are shown in UTC. Let's change it to show time from now like Twitter does.

Install momentjs

We use a time formatting JavaScript library "momenjs" as a client-side library. Install the library using bower.

% bower install --save momentjs

"--save" options saves the package name to "bowser.json", and running "grunt" automatically adds script tags to load the library to "index.html".

Create fromNow AngularJS filter

Create "fromNow" AngularJS filter. Filter is an AngularJS feature to format the value. So, let's create a filter to format time as time from now.

Generate fromNow filter using a generator. The generator will create a directory and put the filter code and test code under the directory. For now, when we created new directory, we need to run "grunt injector" or "grunt serve" to load JavaScript files. ( grunt-contrib-watch/issues/166 )

% yo angular-fullstack:filter fromNow
% grunt injector

Edit the filter code to call momentjs's fromNow() function.

client/app/fromNow/fromNow.filter.js

    return function (input) {
      return moment(input).fromNow();
    };

Now, the fromNow filter has been implemented.

So, let's use the filter on the HTML template. We can use the filter just by adding "|filter" at the end of "{{expression}}" style expression. Now, change from "{{thing.createdAt}}" to "{{thing.createdAt|fromNow}}".

client/app/main/main.html:

        <span style="float: right;">({{thing.createdAt|fromNow}})</span>

Now, the message creation times are formatted as time from now like "~minutes ago".

f:id:paiza:20150706145049p:plain

Edit test code

Now, we need to edit the test code because we edited filter code.

Edit the test code to test so that fromNow filter for the current time returns 'a few seconds ago'.

client/app/fromNow/fromNow.filter.spec.js

  it('return "a few seconds ago" for now', function () {
    expect(fromNow(Date.now())).toBe('a few seconds ago');
  });

Now, run tests, and confirm there is no error.

% grunt test

I18N

To format in user language, use "moment-with-locales.min.js". On "client/index.html", add script tag after "".

client/index.html

      <!-- endbower -->
      <script src="bower_components/momentjs/min/moment-with-locales.min.js"></script>

Change the fromNow filter to use the the browser's language (window.navigator.language).

client/app/fromNow/fromNow.filter.js

    return function (input) {
      return moment(input).locale(window.navigator.language).fromNow();
    };

Now, time is formatted using the browser's language (Ex: "〜分前" in Japanese).

f:id:paiza:20150706145342p:plain

Starred messages

Let's add a feature to star/unstar messages.

On servser-side DB model, add starred user to message schema

Store starred users to messages. On MongoDB, you can store an array as a part of a document. So, we'll store starred users as a part of a message. On the message schema, add "stars" field with array type to store the list of user ObjectIDs.

server/api/thing/thing.model.js:

var ThingSchema = new Schema({
...
  stars: [{
    type: Schema.ObjectId,
    ref: 'User'
  }],
});

Add server-side URL routing

To star/unstar a message, add two APIs(star/unstar) to the server-side URL routing. To allow star/unstar only for authenticated users, add "isAuthenticated" to the routing middleware.

server/api/thing/index.js:

router.put('/:id/star', auth.isAuthenticated(), controller.star);
router.delete('/:id/star', auth.isAuthenticated(), controller.unstar);

Implement server-side API.

Implement star/unstar API. We can use the update() function with "{$push/$pull: {field name: value}}" to insert or remove an item to/from an array inside a document. Implementation for the two APIs are same except for "$push" and "$pull". Call "show()" function at the end of API implementation to return an updated message.

server/api/thing/thing.controller.js:

exports.star = function(req, res) {
  Thing.update({_id: req.params.id}, {$push: {stars: req.user._id}}, function(err, num){
    if (err) { return handleError(res)(err); }
    if(num===0) { return res.send(404).end(); }
    exports.show(req, res);
  });
};

exports.unstar = function(req, res) {
  Thing.update({_id: req.params.id}, {$pull: {stars: req.user._id}}, function(err, num){
    if (err) { return handleError(res)(err); }
    if(num === 0) { return res.send(404).end(); }
    exports.show(req, res);
  });
};

Implement client-side controller

Make star/unstar functions on the client-side controller that just calls the server-side star/unstar APIs. Those two functions are same except for requesting methods("put" and "delete"). Also, add "isMyStart()" function to see whether the current user starred a message or not.

client/app/main/main.controller.js:

    $scope.starThing = function(thing) {
      $http.put('/api/things/' + thing._id + '/star').success(function(newthing){
        $scope.awesomeThings[$scope.awesomeThings.indexOf(thing)] = newthing;
      });
    };
    $scope.unstarThing = function(thing) {
      $http.delete('/api/things/' + thing._id + '/star').success(function(newthing){
        $scope.awesomeThings[$scope.awesomeThings.indexOf(thing)] = newthing;
      });
    };
    $scope.isMyStar = function(thing){
      return Auth.isLoggedIn() && thing.stars && thing.stars.indexOf(Auth.getCurrentUser()._id)!==-1;
    };

Edit HTML template file

Edit the HTML file to add star icons, and call "starThing()" to star on click. If the message is already starred, call "unstarThing()" to unstar the message.

client/app/main/main.html:

<div class="container">
  ...
  <div class="row">
    <div ng-repeat="thing in awesomeThings" class="tweet">
      ...
      <div class="arrow_box col-xs-10">
        <button ng-if="isMyTweet(thing)" type="button" class="close" ng-click="deleteThing(thing)">&times;</button>
        <button ng-if=" isMyStar(thing)" type="button" class="close" ng-click="unstarThing(thing)">
          <span class="glyphicon glyphicon-star" style="color: #CF7C00;" ></span>
        </button>
        <button ng-if="!isMyStar(thing)" type="button" class="close" ng-click="starThing(thing)"  >
          <span class="glyphicon glyphicon-star-empty"></span>
        </button>

Now, we can star/unstar messages.

List user messages, starred messages

f:id:paiza:20150706145600p:plain

Until now, the message list shows all users' messages. Let's add a feature to list only user messages or only starred messages.

Add client-side routing

Create new URLs to show each user's messages or starred messages.

  • User messages: /users/USER-ID
  • Starred messages: /users/USER-ID/starred

Client-side routing is set using "$stateProvider.state" function. Add the above URLs to routing with the same controller ("MainCtrl") and template ("main.html"). To filter messages, we set a query. Add "query" to "resolve" field. Filter by user for user messages, and filter by user ID of "stars" field for starred messages.

On MongoDB, we can write queries using JavaScript. So, we can just transfer the query to MongoDB through the server-side API to filter messages.

Note that if you put "/users/:userId" first on the routing, "starred" will be part of userId. So, put "/users/:userId/stared" before that.

client/app/main/main.js:

angular.module('paizatterApp')
  .config(function ($stateProvider) {
    $stateProvider
      .state('main', {
        url: '/',
        templateUrl: 'app/main/main.html',
        controller: 'MainCtrl',
        resolve: {
          query: function(){return null;}
        },
      })
      .state('starred', {
        url: '/users/:userId/starred',
        templateUrl: 'app/main/main.html',
        controller: 'MainCtrl',
        resolve: {
          query: function($stateParams){
            return {stars: $stateParams.userId};
          }
        }
      })
      .state('user', {
        url: '/users/:userId',
        templateUrl: 'app/main/main.html',
        controller: 'MainCtrl',
        resolve: {
          query: function($stateParams){
            return {user: $stateParams.userId};
          }
        }
      })
      ;
  });

Edit client-side controller

Add a query parameter to the server-API request. Add "query" to the controller function, and add the "query" to "$http.get()" argument.

client/app/main/main.controller.js:

  .controller('MainCtrl', function ($scope, $http, socket, Auth, query) {
  ...
    $http.get('/api/things', {params: {query: query}}).success(function(awesomeThings) {

Edit server-side controller

On the server-side controller, just transfer the received query to MongoDB by passing the query as "find()" arguments.

server/api/thing/thing.controller.js

exports.index = ...
  var query = req.query.query && JSON.parse(req.query.query);
  Thing.find(query).sort...

Add Nav links to Navbar

Until now, Navbar has only one link "Home". Change it to three links like "All", "Mine", and "Starred".

Add link items to $scope.menu array. Enable "Mine" or "Starred" links only for logged-in users. To switch links dynamically before or after login, set the "link" field(for URL) and the "show" field as functions.

client/components/navbar/navbar.controller.js:

    $scope.menu = [
      {
        'title': 'All',
        'link': function(){return '/';},
        'show': function(){return true;},
      },
      {
        'title': 'Mine',
        'link': function(){return '/users/' + Auth.getCurrentUser()._id;},
        'show': Auth.isLoggedIn,
      },
      {
        'title': 'Starred',
        'link': function(){return '/users/' + Auth.getCurrentUser()._id + '/starred';},
        'show': Auth.isLoggedIn,
      },
    ];

On the Navbar HTML file, change from the "link" variable to the "link()" function. Set "item.show()" on "ng-show" to display only when show() returns true.

client/components/navbar/navbar.html:

        <li ng-repeat="item in menu" ng-class="{active: isActive(item.link())}" ng-show="item.show()">
            <a ng-href="{{item.link()}}">{{item.title}}</a>
        </li>

Edit client-side HTML template

Show the user message on click user name. Just add a link to the user message URL (/users/userID).

client/app/main/main.html:

        <a ng-href="/users/{{thing.user._id}}">{{thing.user.name}}</a>

Edit test code

Edit the test code to add a dummy query parameter.

client/app/main/main.controller.spec.js:

    MainCtrl = $controller('MainCtrl', {
      $scope: scope,
      query: null,
    });

Now, we can see my or other users' messages or starred messages.

Search

f:id:paiza:20150706145744p:plain

Let's add a feature to search messages. MongoDB has a full text search feature, so let's use it.

Change client-side URL routing configuration

Use URLs below with "keyword" for searching.

  • All: /?keyword=KEYWORD
  • User: /users/:userId?keyword=KEYWORD
  • Starred: /users/:userId/starred?keyword=KEYWORD

On routing configuration, write to "url" field like "XXX?keyword" so that we can use "keyword" as a parameter.

client/app/main/main.js

      .state('main', {
        url: '/?keyword',
      ...
      .state('starred', {
        url: '/users/:userId/starred?keyword',
      ...
      .state('user', {
        url: '/users/:userId?keyword',

Add search box

Add a search box to the Navbar. Set "search(keyword)" to "ng-submit" attribute so that submitting keyword invoke the search function.

    <div collapse="isCollapsed" class="navbar-collapse collapse" id="navbar-main">
      ...
      <form class="navbar-form navbar-left" role="search" ng-submit="search(keyword)">
        <div class="input-group">
          <input type="text" class="form-control" placeholder="Search" ng-model="keyword">
          <span class="input-group-btn">
            <button type="submit" class="btn btn-default"><span class="glyphicon glyphicon-search" ></span></button>
          </span>
        </div>
      </form>

      <ul class="nav navbar-nav navbar-right">
      ...

Navbar controller: change routing state on search.

On the Navbar controller, add a search function to change the URL state to have the searching keyword specified.

client/components/navbar/navbar.controller.js:

    $scope.search = function(keyword) {
      $state.go('main', {keyword: keyword});        
    };

This works. But, it always searches all messages. It would be nice if we could also restrict the search to user messages or starred messages. Change the "search()" function to keep the URL state (main, user, or starred) on search. Don't forget to add '$state' to the NavbarCtrl function argument to use the variable.

client/components/navbar/navbar.controller.js:

  .controller('NavbarCtrl', function ($scope, $location, Auth, $state) {
    $scope.search = function(keyword) {
      if ($state.current.controller === 'MainCtrl'){
        $state.go($state.current.name, {keyword: keyword}, {reload: true});        
      }else{
        $state.go('main', {keyword: keyword}, {reload: true});        
      }
    };

Regular expression search

Now, let's try to search using a regular expression. Use MongoDB's '$regex' operator to search by the regular expression.

    var keyword = $location.search().keyword;
    if(keyword){
      query = _.merge(query, {name: {$regex: keyword, $options: 'i'}});
    }
    $http.get('/api/things', {params: {query: query}})...

Full-text search

Regular expression search works, but it will become slow if we have many messages.

So, let's use full-text search MongoDB provides. MongoDB have '$text' / '$search' operators for full text search. For full-text search, we don't (can't) specify field to search.

    var keyword = $location.search().keyword;
    if(keyword){
      query = _.merge(query, {$text: {$search: keyword}});
    }
    $http.get('/api/things', {params: {query: query}})...

Edit server-side model

For the full-text search, add a 'text' index to the searching field on the schema.

server/api/thing/thing.mode.js:

ThingSchema.index({name: 'text'});

Now, we can search by word like "Development". (We cannot search by substring match.)

Japanese search

MongoDB's full-text searching only supports Latin languages, it and does not support other languages as Japanese.

We can use ElasticSearch or other engines. But for now, we will use "TinySegmenter" to tokenize Japanese.

Add "tokenizedName" field to store and index tokenized messages.

var ThingSchema = new Schema({
  name: String,
  tokenizedName: String,
  ...
}
...
ThingSchema.index({tokenizedName: 'text', name: 'text'});

Drop "things" collection for re-indexing.

% mongo
> use APPLICATION-dev
> db.things.drop()

Install TinySegmenter using npm as a server-side library.

% npm install --save r7kamura/tiny-segmenter

Tokenize a message on save, and join with space, and save to the "tokenizedName" field. You can hook on save by calling "pre('save', callback)" on the schema.

server/api/thing/thing.model.js:

var TinySegmenter = require('tiny-segmenter');
...
ThingSchema.pre('save', function(next){
  var tinySegmenter = new TinySegmenter();
  this.tokenizedName = tinySegmenter.segment(this.name).join(' ');
  next();
});

Now, we can search by Japanese words. For example, we can search the message "吾輩は猫である" for the word "我輩".

Infinite scroll

Now, we can only see the last 20 messages, and there is no way to see older messages.

Let's add an infinite scroll to see older messages, like Twitter does.

On client-side, install "ngInfiniteScroll" library

Install an AngularJS library "ngInfiniteScroll" for an infinite scroll.

% bower install --save ngInfiniteScroll
% grunt wiredep

Load ngInfiniteScroll

To use the ngInfiniteScroll module, add it to the AnguarJS application module dependency.

client/app/app.js:

angular.module('paizatterApp', [
   ... ,
   'infinite-scroll'
]);

Edit HTML file

To use the ngInfiniteScroll module, on div tag of "container" class, add "infinite-scroll" attribute to call a function ("nextPage()") on scroll. Set flags ("busy", "noMoreData") to "infinite-scroll-disabled" attribute not to scroll while loading or if no more messages are available.

At the end of the HTML file, output "Loading data" while loading.

client/app/main/main.html:

<div class="container" infinite-scroll='nextPage()' infinite-scroll-disabled='busy || noMoreData'>
  ...
  <div ng-show='busy'>Loading data...</div>
</div>

Edit client-side controller

On the "$scope" variable, create "busy" field to store for the loading state, and "noMoreData" flag to store whether all the message is loaded or not.

On scroll, we need to load messages older than the last message. Add "{_id: {$lt: lastId}}" to the query.

On initial loading, if there are messages fewer than 20, set "noMoreData" flag.

client/app/main/main.controller.js:

    $scope.busy = true;
    $scope.noMoreData = false;
    ...
    $http.get('/api/things', ...
      ...
      if($scope.awesomeThings.length<20){
        $scope.noMoreData = true;
      }
      $scope.busy = false;
    });

    $scope.nextPage = function(){
      if($scope.busy){
        return;
      }
      $scope.busy = true;
      var lastId = $scope.awesomeThings[$scope.awesomeThings.length-1]._id;
      var pageQuery = _.merge(query, {_id: {$lt: lastId}});
      $http.get('/api/things', {params: {query: pageQuery}}).success(function(awesomeThings_) {
        $scope.awesomeThings = $scope.awesomeThings.concat(awesomeThings_);
        $scope.busy = false;
        if(awesomeThings_.length === 0){
          $scope.noMoreData = true;
        }
      });
    };

Now, we can load messages older than the last 20 messages, using infinite scroll.

Edit karma.conf

Add "ngInfiniteScroll" library to karma.conf to load on test.

karma.conf.js:

    files: [
      ...
      'client/bower_components/ngInfiniteScroll/build/ng-infinite-scroll.js',
      ...
    ]

Check test result.

% grunt test

Re-deploy

Finally, we have built all the features! Let's re-deploy the latest version to Heroku.

% grunt
% grunt buildcontrol:heroku

Open browser and see it works!

http://APPLICATION.herokuapp.com/

Summary

In this article, we created a Twitter-like full-stack web service using a MEAN stack, Angular Full-Stack generator. Although we have not edited many lines of codes, we have build a nearly full-fledged, real web service.

With MEAN stack, we can easily create web services just using JavaScript. Let's come up with ideas and build your own web services!

I welcome your feedback about the instruction.

I'll continue writing articles about creating web services using MEAN stack.

MEAN stack development articles
Building full-stack web service - MEAN stack development(1)
* Building Twitter-like full-stack web service in 1 hour - MEAN stack development (2)
Building a QA web service in an hour - MEAN stack development(3)

Building full-stack web service - MEAN stack development(1)

(Japanese article is here.)

f:id:paiza:20140712194904j:plain (by Yoshioka Tsuneo, @ at https://paiza.IO/)

Nowadays, it is getting hard to build web services because we need to use full-stack environment: browser(client) code for interactive UI, server code for shared data or logic, and WebSockets for real-time communication.

MEAN stack package frameworks from client side to server side, making web service development much simpler, easier, and faster.

In this article, I'll introduce one of MEAN stack implementation, AngularJS Full-Stack generator(generator-angular-fullstack), and actually run, edit, and deploy an application.

There are many articles about development only for client, or only for server. But, few mention how to mash up those individual tools. Thanks to MEAN stack, now we have a best practice to build full stack applications without bothering mashing up these components. Let's try for the cool environment !

In the following articles, based on this article, I'll also introduce how to build more practical applications.

f:id:paiza:20150703174202p:plain

AngularJS Full-stack generator Demo:http://fullstack-demo.herokuapp.com

Contents

What is MEAN stack ?

MEAN stack is a package of frameworks, MongoDB, Express, AngularJS, and Node.js, to build full-stack web services.

https://www.mongodb.org

http://expressjs.com/

https://angularjs.org

https://nodejs.org/

In the past, web service frameworks, such as CakePHP and Ruby on Rails, provided best practices to build web services easily, quickly, and securely.

But, those frameworks are based on the basic HTTP flow in which clients send requests to servers, servers generate and send HTML pages to clients, and clients render the HTML in browsers. Form action or following links completely discards the current page and renders new pages generated on the servers.

However, this request-response-based architecture causes delays in every action, and clear all states(inputs, selection scrolling, etc...), and does not provide interactive UI.

So, to make web more interactive, web services like Google Maps started using JavaScript-based Ajax technology.

At first, JavaScript was used only on specific UI parts. But, more and more parts were built using JavaScript, and eventually JavaScript framework is demanded to build web application. AngularJS is one of the most widely-used (client side) JavaScript framework.

Furthermore, request-response style architecture prevents web services from actively sending notifications to users. There are few ways to send new message notifications on chat applications, or display other players' moves in the multiplayer games, for example. WebSocket technology solves such restrictions on the web with full-duplex persistent connections. With WebSocket, because browser and server applications are persistently connected, server applications need to handle a bunch of connections at the same time. Node.js manages multiple clients(browsers) on a single thread, utilizing asynchronous operations JavaScript have.

So now, we can use JavaScript for both client side, and server side. Now, it is time to introduce JavaScript database, MongoDB! MongoDB is schema-less, so it's the best fit for rapid development.

MEAN stack is such a JavaScript based full-stack environment combining the client side framework AngularJS, server side framework Node.js/Express.JS, and database MongoDB.

MEAN stack itself is a name for combination. Actual implementation includes MEAN.IO or MEAN.JS. In this article, I'll introduce AngularJS Full-stack generator(generator-angular-stack) that is easy to start with sophisticated generators and templates.

MEAN stack features(generator-angular-fullstack)

Full-stack

Full-stack web development environments provides a best practice for clients, servers and databases. That makes easy to understand whole project structure. It is especially convenient to have preconfigured combinations among clients, servers, databases, etc...

Only one language: JavaScript

We can use one language, JavaScript, to develop whole projects including clients, servers and databases. It makes development quite stress-free because you don't no need to switch context between languages.

Based on common tools

Frameworks used in MEAN stack, MongoDB, Node.js, Express, or AngularJS, are not used solely for MEAN stack. Individual frameworks are just as widely used. Tools used in AngularJS Full-stack generator, Yeoman, Bower, npm, Grunt, Karma are also commonly-used tools.

These common frameworks or tools introduce development environment stablity, use cases, and knowledges. Web development environments changes from time to time, so it is important to use common tools in order to be flexible and catch up the latest technologies.

Generator

AngularJS Full-stack generator can generate project templates using a menu-style interface. AngularJS Full-stack generator supports the following configurations:

  • Client

    • Scripts: JavaScript, CoffeeScript, Babel
    • Markup: HTML, Jade
    • Stylesheets: CSS, Stylus, Sass, Less
    • AnguarJS Routers: ngRoute, ui-router
  • Server

    • Database: None, MongoDB
    • Authentication boilerplate: Yes, No
    • oAuth integrations: Facebook, Twitter, Google
    • Socket.io integrations: Yes, No

AngularJS Full-Stack generator also provides generators for client-side and server-side code modules. These generators make it easy to start development. Heroku or OpenShift deployment commands are also provided to release products easily.

Install

So, let's install the MEAN stack implementation, AngularJS Full-Stack generator. MEAN stack works on Mac OS X, Linux, or Windows. In this article, I'm using Mac OS X, but most of the following commands works just as well on other OS.

  • Install Node.js (If not installed)

Browse https://nodejs.org, and click "Install" to download, then install Node.js package.

  • Install Yeoman, Bower, Grunt, Gulp (If not installed)
% sudo npm install -g yo bower grunt-cli gulp
  • Install AngularJS Full-Stack generator(generator-angular-fullstack)
% sudo npm install -g generator-angular-fullstack
  • Install Homebew (If not installed)
% ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  • Install MongoDB(If not installed)
% brew update
% brew install mongodb
% ln -fs /usr/local/opt/mongodb/homebrew.mxcl.mongodb.plist ~/Library/LaunchAgents/
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mongodb.plist
  • Test MongoDB

MongoDB has a flexible schema, and does not have a "create" statement to declare schema, but you can just insert data directly into the database.

In MongoDB, each data("record" in SQL) is called a "document" which stores any JSON objects, and the collection of data("table" in SQL) is just called "collection".

So now, let's try to insert, find, or delete documents.

~$ /usr/local/bin/mongo
MongoDB shell version: 3.0.2
connecting to: test
> db.my_collection.save( { 'hello' : 'world' } )
WriteResult({ "nInserted" : 1 })
> db.my_collection.find()
{ "_id" : ObjectId("558b92df5eed5e30d6c9c84f"), "hello" : "world" }
> db.my_collection.remove({})
WriteResult({ "nRemoved" : 1 })

GUI tools like MongoHub on Mac OS X also exist to make database operation easy.

f:id:paiza:20150706155509p:plain

MongoHub

Create a new project

Now, let's create a new project.

  • Make project directory
% mkdir sample
% cd sample
  • Generate project files using AngularJS Full-Stack generator
% yo angular-fullstack sample

You will encounter the following question (below). You can just choose the default options, but maybe nice to enable oAuth integration. For now, let's enable all social networks.

- Would you like to include additional oAuth strategies? 
 ◉ Google
 ◉ Facebook
❯◯ Twitter

After you answer all questions, project files are generated, and modules are installed. It may take some time for the first time. Just have a cup of coffee.

  • Updating npm packages Default npm packages on Angular Full-stack generator is a bit old, so let's update to the latest versions using npm-check-updates .
% npm install -g npm-check-updates
% npm-check-updates -u
% npm install

Start server

It is time to start the server. Just type a command like this:

% grunt serve

This will setup all tasks, and start the Node.js server, and opens the browser with URL: http://localhost:9000 .

At this point, the following features are already implemented:

  • Twitter Bootstrap based UI
  • Listing, adding, and removing items.
  • Real-time listing update on other user's adding or removing items.
  • User authentication(Sign Up, Login, SNS integration(with API key))

Now, type some words in the input box, and click "Add New". The item listing will then be updated. If you opens multiple browsers and add item, other browsers listing is automatically updated without manually reloading browser.

Edit code

Change the file contents(Keeping "grunt serve" running).

app/main/main.html:

<h1>How's it going ?</h1>

Just after saving the file, (without manually reloading the browser) the browser content is updated in real time.

Debug

Client-side debugging

You can use web browser development tools for debugging.

On Chrome, click the Chrome menu icon at the right of URL base, and choose [More tools][Developer tools]. Then you'll see the "Developer Tools" window.

f:id:paiza:20150708140450p:plain

On Safari, go Safari menu's Preference, check "Show development menu" to enable the development menu. Open a page to debug, go to main menu and choose "Development" - "Show error console" to access development tools.

In the development tools, you can see console logs, set breakpoints in JavaScript code, or print variables.

Server-side debugging

Let's try Node.js debugger(Node Inspector) for server-side code. Node Inspector runs on Chrome. So, update the "open" function on "Gruntfile.js" as shown below to specify the browser application as "Google Chrome.app".

Gruntfile.js:

            // opens browser on initial server start
            nodemon.on('config:update', function () {
              setTimeout(function () {
                require('open')('http://localhost:8080/debug?port=5858', '/Applications/Google Chrome.app');
              }, 500);
            });

Then, run Node.js server on debug mode:

% grunt serve:debug

Now, chrome developer tools will start debugging the Node.js application. The debugger is suspended on the first line of the Node.js application. Just click the "Resume" button(or "F8") to restart the application.

In this window, you can set the breakpoint, step in/over, or see variables.

f:id:paiza:20150706114630p:plain

Deployment

Because "grunt serve" starts the server on the local machine, other users cannot use your web service. To make it public, deploy the application on the public server.

Angular Full-Stack generator supports Heroku deployment.

Create Heroku account

Heroku provides a free plan to deploy one application. Just browse Heroku and click "Sign up for free" to create a new account.

https://www.heroku.com

f:id:paiza:20150706114938p:plain

Install Heroku Toolbelt

To manage application on Heroku, you will use the Heroku Toolbelt command line toolkit. Download and install the toolkit from the URL below.

https://toolbelt.heroku.com

Now, login using the "heroku" command.

% heroku login

Type your Heroku username and password.

Set up application deployment environment for Heroku

To set up the Heorku application deployment environment, use the Angular Full-stack generator command "yo angular-fullstack:heroku".

Type the application name when prompted. The application URL is http://APPLICATION.herokuapp.com .

You'll also need to install the MongoDB module. Heroku supports MongoHQ and MongoLab. For now, let's use MongoLab because of its free plan.

% yo angular-fullstack:heroku
% cd dist
% heroku addons:add mongolab

From now on, you can build and deploy the application using the "grunt" and "grunt buildcontrol:heroku" commands.

% grunt
% grunt buildcontrol:heroku

Done !

Let's access the application URL with your browser.

http://APPLICATION.herokuapp.com/

In case you encounter any errors, review log messages.

% heroku logs

SNS integration

Although Sign Up and Login are already implemented, you need to set up both API key and SECRET key to integrate SNS authentication.

Twitter

On https://apps.twitter.com , click "Create New App" to create a new application. Specify the application URL to "Callback URL"(Ex: http://paizatter.herokuapp.com ).

On the application page, see "Keys and Access Tokens" to access "Consumer Key (API Key)" and "Consumer Secret (API Secret)".

f:id:paiza:20150706115258p:plain

Facebook

On https://developers.facebook.com/apps/ , click "Add a New App". Choose "Website" and input your application name and choose the category like "Utilities" under "Choose a Category". Then click "Create App ID" to create.

Choose your application from the "My Apps" menu. Choose the "Advanced" tab and specify the application URL on "Valid OAuth redirect URIs" (Ex: http://APPLICATION.herokuapp.com ). You can specify multiple URLs, so adding "http://localhost:9000/" will be convenient for testing on local environment.

f:id:paiza:20150706115559p:plain

From application's "Settings" menu, input "Contact Email". Choose "Status & Review" and answer "Yes" on "Do you want to make this app and all its live features available to the general public?" to enable application. Now, you can get access Dashboard to get "App ID" and "App Secret".

Google

On https://console.developers.google.com/project , click "Create project". Choose your created project. From the menu on the left, choose "APIs&auth"/"Credentials". On the OAuth page, click "Create new Client ID" and choose "Web application" and click "Configure consent screen".

After the consent, type the application URL under "Authorized redirect URIs" (Ex: http://APPLICATION.herokuapp.com/auth/google/callback ), and click "Create Client ID".

f:id:paiza:20150706124704p:plain

From "APIs&auth"-"API" menu, choose "Google+ API" and enable "Google+ API". Now, you can see "APIs&auth"/"Credentials" to access Client ID(API key) and Client secret(SECRET key).

Set up API key on Heroku

After retrieving API keys and SECRET keys, use the "heroku config set" command to set keys on environment variables.

% cd dist
% heroku config set FACEBOOK_ID=xxx
% heroku config set FACEBOOK_SECRET=xxx
% heroku config set TWITTER_ID=xxx
% heroku config set TWITTER_SECRET=xxx
% heroku config set GOOGLE_ID=xxx
% heroku config set GOOGLE_SECRET=xxx
% heroku config
% heroku restart

Summary

In this article, I introduced how you can use the AngularJS Full-Stack generator to build, run, and deploy MEAN stack web services. MEAN stack makes it easy to build full-stack web services quickly. So, it is especially convenient for project startup or prototyping. Let's try it !

Note: This article just runs asample application. The following articles will provides more practical applications.

Reference

AngularJS Full-Stack generator

https://github.com/DaftMonk/generator-angular-fullstack

MEAN stack development articles
* Building full-stack web service - MEAN stack development(1)
Building Twitter-like full-stack web service in 1 hour - MEAN stack development (2)
Building a QA web service in an hour - MEAN stack development(3)