how to disassemble one monster app into an ecosystem of 30

From 1 To 30How To Disassemble

One Monster App Into An Ecosystem Of 30

Jonathan Palley, CTO/COO

Guo Lei, Chief Architect

© 2010 Idapted, Ltd.

Beta 技术沙龙http://club.blogbeta.com

官方twitter : @betasalon

Groups : http:// groups.google.com/group/betasalon

An

Experience

A Tale of Two Buildings

2009 ShanghaiLotus Riverside Community

1909 BeijingForbidden City

1

30

What is one?

The entire web application/system/platform

runs as one single rails application

(We are talking about really large systems. Multiple different types of clients/functions)

Problems

Hard to test/extend/scale

Confused new staff

What is 30?

A ecosystem of applications

Independent

Linked and Seamless

Basic features of each app• Separate database• Runs independently (complete story) • Lightweight (single developer)• Tight internal cohesion and loose external coupling

Advantages

• Independent Development Cycle

• Developer autonomy

• Technology (im)maturity safety

APPEAL TO DEVELOPER LAZINESS

What’s the mystery of the forbidden city?

Consistent UI

• Shared CSS/JS/Styleguide

• Common Helpers in Shared Gem

• Safely try new things

All applications use the same base CSS/JS

Keep all the application the same style

<%= idp_include_js_css %>

# =><script src ="/assets/javascripts/frame.js" type="text/javascript"></script>

<link href="/assets/stylesheets/frame.css" media="screen" rel="stylesheet" type="text/css" />

interface

CSS Framework

interface

Abstract Common Helpers to Gem

Search function for models

interface

Common Helpers: Combo search (cont)

View:<%= search_form_for(HistoryRecord, :interaction_id, :released,[:rating, {:collection=>assess_ratings}],[:mark_spot_num,{:range=>true}], [:created_at, {:ampm=>true}]) %>

Controller:@history_records = HistoryRecord.combo_search(params)

interface

Common Helpers: List table

well formattedwith pagination

sortablecustomizable

interface

Common Helpers: List table (cont)

<%=idp_table_for(@history_records,:sortable=>true,:customize =>

"history_records") do |item, col|col.add :id, link_to(item.id, admin_history_record_path(item)),:order=>:idcol.build :duration, :waiting_time, :review_timecol.add :scenario, item.scenario_title, :order => :scenario_titlecol.add :mark_spot_num

end

%>

interface

Development Lifecycle

interface

1. Implement new View code/plugin in a second application

2. Abstract into plugin using existing “idp” helpers

3. Put it into main view gem

interface data

How do applications share data?

(remember: each app has its own data)

data

-“Read Only” Database Connections- Services- AJAX Loaded View Segments

Business example

user

course

purchase

learning process

data

Purchase App

Requirement: List course packages for user to select to purchase

The course package data is stored in the “course” application

but

data

Solution

readonly db connection

data

course

Code

Model:class CoursePackage < ActiveRecord::Base

acts_as_readonly :courseend

View:<ul><% CoursePackage.all.each do |package| %>

<li><%= package.title %> <%= package.price %></li><% end %></ul>

data

Why doesn’t this break the rule of loose coupling?

Model:class CoursePackage < ActiveRecord::Base

acts_as_readonly :courseend

View:<ul><% CoursePackage.all.each do |package| %>

<li><%= package.title %> <%= package.price %></li><% end %></ul>

data

acts_as_readonly in Depth

def acts_as_readonly(name, options = {})

config = CoreService.app(name).database

establish_connection config[Rails.env]

set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s)

end

data

acts_as_readonly in Depth

def acts_as_readonly(name, options = {})

config = CoreService.app(name).database

establish_connection config[Rails.env]

set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s)

end

data

Core service

class CoreService < ActiveResource::Base

self.site = :user

def self.app(app_name)

CoreService.find(app_name)

end

end

data

Centralized configuration

data

How does Core know all the configurations?

Each app posts its configuration to core when it is started

data

data

config/site_config.yml

app: course

api:

course_list: package/courses

config/initializers/idp_initializer.rb

CoreService.reset_config

data

core_service.rb in idp_lib

APP_CONFIG = YAML.load(Rails.root.join(“config/site_config.yml”))

def reset_config

self.post(:reset_config, :app => {

:name => APP_CONFIG["app"],

:settings => APP_CONFIG,

:database => YAML.load_file(

Rails.root.join("config/database.yml"))})

end

data

Model in Purchase:class CoursePackage < ActiveRecord::Base

acts_as_readonly :courseend

Again, implemented in gem

data

config/environment.rb

config.gem ‘idp_helpers’

config.gem ‘idp_lib’

data

gems

• Web services for “write” interactions

class CoursePackageService < ActiveSupport::Base

self.site = :course

end

data

example

Roadmap needs to be generated after learner pays.

data

codesCourse: app/controllers/roadmap_services_controller.rb

def create

Roadmap.generate(params[:user_id], params[:course_id])

end

Purchase: app/models/roadmap_service.rb

class RoadmapService < ActiveSupport::Base

self.site = :course

end

Purchase: app/models/order.rb

def activate_roadmap

RoadmapService.create(self.user_id, self.course_id)

end

data

AJAX Loaded Composite Viewdata

<div>

<%= ajax_load(url_of(:course, :course_list)) %>

</div>

Fetched from different

applications

Ecosystem url_for

Open Source

http://github.com/idapted/eco_apps

data

interface data user

Features of User Service

• Registration/login

• Profile management

• Role Based Access Control

user

Access Control

Each Controller is one Node

user

* Posted to user service when app starts

Access Control

before_filter :check_access_right

def check_access_right

unless xml_request? or inner_request?

access_denied unless has_page_right?(params[:controller])

end

end

user

* Design your apps so access control can be by controller!

How to share?

user

Step 1: User Auth

SSO

Shared Session

Same Session Store

user

Step 1: User Auth

config/initializers/idp_initializer.rb

ActionController::Base.session_store = :active_record_store

ActiveRecord::SessionStore::Session.acts_as_remote :user,

:readonly => false

user

Step 2: Access Control

Tell core its controllers structure

CoreService. reset_rights

def self.reset_rights

data = load_controller_structure

self.post(:reset_rights, :data => data)

end

user

Step 2: Access Control

before_filter :check_access_right

def check_access_right

unless xml_request? or inner_request?

access_denied unless has_page_right?(params[:controller])

end

end

user

Step 2: Access Control

has_page_right?

Readonly db conn again

user

Step 2: Access Control

def has_page_right?(page)

roles = current_user.roles

roles_of_page = IdpRoleRight.all(:conditions => ["path = ?", page]).map(&:role_id)

(roles - (roles - roles_of_page)).size > 0

end

class IdpRoleRight < ActiveRecord::Base

acts_as_readonly :user, :table_name => "role_rights"

end

user

Again, gems!user

config/environment.rb

config.gem ‘idp_helpers’

config.gem ‘idp_lib’

config.gem ‘idp_core’

interface servicedata user

Support applications

• File

• Mail

• Comet service

service

File

class Article < ActiveRecord::Basehas_files

end

@article.files.first.url

Upload File in Background to

FileService

Store with app_name,

model_name, model_id

Use readonlymagic to easily

display

Idp_file_form

Specify Class that Has Files

Comet

service

class ChatRoom < ActiveRecord::Baseacts_as_realtime

end

<%= realtime_for(@chat_room, current_user.login) %>

<%= realtime_data(dom_id, :add, :top) %>

@chat_room.realtime_channel.broadcast(“hi world", current_user.login)

mail

Mail services

MailService.send(“test@idapted.com”, :welcome,

:user => “test”)

service

Host all in one domain

Load each rails app into a subdir, we use Unicorn

unicorn_rails --path /userunicorn_rails --path /studycenterunicorn_rails --path /scenario

Host all in one domain

use Nginx as a reverse proxy

location /user {proxy_pass http://rails_app_user;

}

location /studycenter {proxy_pass http://rails_app_studycenter;

}

Host all in one domain

All you see is a uniform URL

www.eqenglish.com/user

www.eqenglish.com/studycenterwww.eqenglish.com/scenario

Pair-deploy

How to split one into many?

By Story

Each App is one group of similar features.

By DataEach App writes to the same data

Example

• User Management

• Course package

• Purchase

• Learning process

• …

Iteration

Be adventurous at the beginning.

Split one into as many as you think is sensitive

Then you may find

• Some applications interact with each other frequently.

• Lots of messy and low efficiency code to deal with interacting.

Merge them into one.

Measurement

• Critical and core task of single app should not call services of others.

• One doesn’t need to know much about others’ business to do one task (or develop).

• Independent Stories

Pitfalls

• Applications need to be on the

same intranet.

• No “right place” for certain cases.

Results: Higher Productivity

-Faster build to deploy-More developer autonomy-Safer- Scalable-Easier to “jump in”- Greater Happiness

Support Tech

• FreeSWITCH

• VoIP

• http://www.freeswitch.org.cn

• Flex

• Erlang

• concurrent tasks

• Titanium

• mobile

Full Stack of Us

Rails/FlexCTO

VoIP UI SystemUE

Full Stack of Us

Develop environment:

• 4 mbp + 4 black apples + 1 linux

• textmate & netbeans

Servers:• test: 2 physical pc with Xen serving 10

virtual servers

• production: 2 (China) + 5(US)

Full Stack of Us

Web Server:

• nginx + unicorn

Rails:

• Rails version 2.3.4

• Ruby version: 1.8.7

Full Stack of Us

Plugins:

• exception_notification

• will_paginate

• in_place_editor_plus

• acts_as_list

• open_flash_chart

• Spawn + workling

Full Stack of Us

Gems:

• rspec

• factory_girl

• thinking-sphinx

DB:

• mysql

Cache:

• memcache

Full Stack of Us

Test:

• rspec + factory_girl

• Autospec

Deploy:

• webistrano

Full Stack of Us

Team Build:

• board

• code review

• tea break

• retreat

Join Us

If you are:

• smart

• creative

• willing to do great things with great people

• happen to know JS/Flex/Rails

Send whatever can prove your ability to:

tech-team-jobs@idapted.com

© 2010 Idapted, Ltd.

Thank you!

Q&A

http://developer.idapted.com

jpalley@idapted.com (@jpalley)

guolei@idapted.com (@fiyuer)