how to disassemble one monster app into an ecosystem of 30
TRANSCRIPT
From 1 To 30How To Disassemble
One Monster App Into An Ecosystem Of 30
Jonathan Palley, CTO/COO
Guo Lei, Chief Architect
© 2010 Idapted, Ltd.
Beta 技术沙龙http://club.blogbeta.com
官方twitter : @betasalon
Groups : http:// groups.google.com/group/betasalon
An
Experience
A Tale of Two Buildings
2009 ShanghaiLotus Riverside Community
1909 BeijingForbidden City
1
30
What is one?
The entire web application/system/platform
runs as one single rails application
(We are talking about really large systems. Multiple different types of clients/functions)
Problems
Hard to test/extend/scale
Confused new staff
What is 30?
A ecosystem of applications
Independent
Linked and Seamless
Basic features of each app• Separate database• Runs independently (complete story) • Lightweight (single developer)• Tight internal cohesion and loose external coupling
Advantages
• Independent Development Cycle
• Developer autonomy
• Technology (im)maturity safety
APPEAL TO DEVELOPER LAZINESS
What’s the mystery of the forbidden city?
Consistent UI
• Shared CSS/JS/Styleguide
• Common Helpers in Shared Gem
• Safely try new things
All applications use the same base CSS/JS
Keep all the application the same style
<%= idp_include_js_css %>
# =><script src ="/assets/javascripts/frame.js" type="text/javascript"></script>
<link href="/assets/stylesheets/frame.css" media="screen" rel="stylesheet" type="text/css" />
interface
CSS Framework
interface
Abstract Common Helpers to Gem
Search function for models
interface
Common Helpers: Combo search (cont)
View:<%= search_form_for(HistoryRecord, :interaction_id, :released,[:rating, {:collection=>assess_ratings}],[:mark_spot_num,{:range=>true}], [:created_at, {:ampm=>true}]) %>
Controller:@history_records = HistoryRecord.combo_search(params)
interface
Common Helpers: List table
well formattedwith pagination
sortablecustomizable
interface
Common Helpers: List table (cont)
<%=idp_table_for(@history_records,:sortable=>true,:customize =>
"history_records") do |item, col|col.add :id, link_to(item.id, admin_history_record_path(item)),:order=>:idcol.build :duration, :waiting_time, :review_timecol.add :scenario, item.scenario_title, :order => :scenario_titlecol.add :mark_spot_num
end
%>
interface
Development Lifecycle
interface
1. Implement new View code/plugin in a second application
2. Abstract into plugin using existing “idp” helpers
3. Put it into main view gem
interface data
How do applications share data?
(remember: each app has its own data)
data
-“Read Only” Database Connections- Services- AJAX Loaded View Segments
Business example
user
course
purchase
learning process
data
Purchase App
Requirement: List course packages for user to select to purchase
The course package data is stored in the “course” application
but
data
Solution
readonly db connection
data
course
Code
Model:class CoursePackage < ActiveRecord::Base
acts_as_readonly :courseend
View:<ul><% CoursePackage.all.each do |package| %>
<li><%= package.title %> <%= package.price %></li><% end %></ul>
data
Why doesn’t this break the rule of loose coupling?
Model:class CoursePackage < ActiveRecord::Base
acts_as_readonly :courseend
View:<ul><% CoursePackage.all.each do |package| %>
<li><%= package.title %> <%= package.price %></li><% end %></ul>
data
acts_as_readonly in Depth
def acts_as_readonly(name, options = {})
config = CoreService.app(name).database
establish_connection config[Rails.env]
set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s)
end
data
acts_as_readonly in Depth
def acts_as_readonly(name, options = {})
config = CoreService.app(name).database
establish_connection config[Rails.env]
set_table_name(self.connection.current_database + (options[:table_name]||table_name).to_s)
end
data
Core service
class CoreService < ActiveResource::Base
self.site = :user
def self.app(app_name)
CoreService.find(app_name)
end
end
data
Centralized configuration
data
How does Core know all the configurations?
Each app posts its configuration to core when it is started
data
data
config/site_config.yml
app: course
api:
course_list: package/courses
config/initializers/idp_initializer.rb
CoreService.reset_config
data
core_service.rb in idp_lib
APP_CONFIG = YAML.load(Rails.root.join(“config/site_config.yml”))
def reset_config
self.post(:reset_config, :app => {
:name => APP_CONFIG["app"],
:settings => APP_CONFIG,
:database => YAML.load_file(
Rails.root.join("config/database.yml"))})
end
data
Model in Purchase:class CoursePackage < ActiveRecord::Base
acts_as_readonly :courseend
Again, implemented in gem
data
config/environment.rb
config.gem ‘idp_helpers’
config.gem ‘idp_lib’
data
gems
• Web services for “write” interactions
class CoursePackageService < ActiveSupport::Base
self.site = :course
end
data
example
Roadmap needs to be generated after learner pays.
data
codesCourse: app/controllers/roadmap_services_controller.rb
def create
Roadmap.generate(params[:user_id], params[:course_id])
end
Purchase: app/models/roadmap_service.rb
class RoadmapService < ActiveSupport::Base
self.site = :course
end
Purchase: app/models/order.rb
def activate_roadmap
RoadmapService.create(self.user_id, self.course_id)
end
data
AJAX Loaded Composite Viewdata
<div>
<%= ajax_load(url_of(:course, :course_list)) %>
</div>
Fetched from different
applications
Ecosystem url_for
Open Source
http://github.com/idapted/eco_apps
data
interface data user
Features of User Service
• Registration/login
• Profile management
• Role Based Access Control
user
Access Control
Each Controller is one Node
user
* Posted to user service when app starts
Access Control
before_filter :check_access_right
def check_access_right
unless xml_request? or inner_request?
access_denied unless has_page_right?(params[:controller])
end
end
user
* Design your apps so access control can be by controller!
How to share?
user
Step 1: User Auth
SSO
Shared Session
Same Session Store
user
Step 1: User Auth
config/initializers/idp_initializer.rb
ActionController::Base.session_store = :active_record_store
ActiveRecord::SessionStore::Session.acts_as_remote :user,
:readonly => false
user
Step 2: Access Control
Tell core its controllers structure
CoreService. reset_rights
def self.reset_rights
data = load_controller_structure
self.post(:reset_rights, :data => data)
end
user
Step 2: Access Control
before_filter :check_access_right
def check_access_right
unless xml_request? or inner_request?
access_denied unless has_page_right?(params[:controller])
end
end
user
Step 2: Access Control
has_page_right?
Readonly db conn again
user
Step 2: Access Control
def has_page_right?(page)
roles = current_user.roles
roles_of_page = IdpRoleRight.all(:conditions => ["path = ?", page]).map(&:role_id)
(roles - (roles - roles_of_page)).size > 0
end
class IdpRoleRight < ActiveRecord::Base
acts_as_readonly :user, :table_name => "role_rights"
end
user
Again, gems!user
config/environment.rb
config.gem ‘idp_helpers’
config.gem ‘idp_lib’
config.gem ‘idp_core’
interface servicedata user
Support applications
• File
• Comet service
service
File
class Article < ActiveRecord::Basehas_files
end
@article.files.first.url
Upload File in Background to
FileService
Store with app_name,
model_name, model_id
Use readonlymagic to easily
display
Idp_file_form
Specify Class that Has Files
Comet
service
class ChatRoom < ActiveRecord::Baseacts_as_realtime
end
<%= realtime_for(@chat_room, current_user.login) %>
<%= realtime_data(dom_id, :add, :top) %>
@chat_room.realtime_channel.broadcast(“hi world", current_user.login)
Host all in one domain
Load each rails app into a subdir, we use Unicorn
unicorn_rails --path /userunicorn_rails --path /studycenterunicorn_rails --path /scenario
Host all in one domain
use Nginx as a reverse proxy
location /user {proxy_pass http://rails_app_user;
}
location /studycenter {proxy_pass http://rails_app_studycenter;
}
Host all in one domain
All you see is a uniform URL
www.eqenglish.com/user
www.eqenglish.com/studycenterwww.eqenglish.com/scenario
Pair-deploy
How to split one into many?
By Story
Each App is one group of similar features.
By DataEach App writes to the same data
Example
• User Management
• Course package
• Purchase
• Learning process
• …
Iteration
Be adventurous at the beginning.
Split one into as many as you think is sensitive
Then you may find
• Some applications interact with each other frequently.
• Lots of messy and low efficiency code to deal with interacting.
Merge them into one.
Measurement
• Critical and core task of single app should not call services of others.
• One doesn’t need to know much about others’ business to do one task (or develop).
• Independent Stories
Pitfalls
• Applications need to be on the
same intranet.
• No “right place” for certain cases.
Results: Higher Productivity
-Faster build to deploy-More developer autonomy-Safer- Scalable-Easier to “jump in”- Greater Happiness
Support Tech
• FreeSWITCH
• VoIP
• http://www.freeswitch.org.cn
• Flex
• Erlang
• concurrent tasks
• Titanium
• mobile
Full Stack of Us
Rails/FlexCTO
VoIP UI SystemUE
Full Stack of Us
Develop environment:
• 4 mbp + 4 black apples + 1 linux
• textmate & netbeans
Servers:• test: 2 physical pc with Xen serving 10
virtual servers
• production: 2 (China) + 5(US)
Full Stack of Us
Web Server:
• nginx + unicorn
Rails:
• Rails version 2.3.4
• Ruby version: 1.8.7
Full Stack of Us
Plugins:
• exception_notification
• will_paginate
• in_place_editor_plus
• acts_as_list
• open_flash_chart
• Spawn + workling
Full Stack of Us
Gems:
• rspec
• factory_girl
• thinking-sphinx
DB:
• mysql
Cache:
• memcache
Full Stack of Us
Test:
• rspec + factory_girl
• Autospec
Deploy:
• webistrano
Full Stack of Us
Team Build:
• board
• code review
• tea break
• retreat
Join Us
If you are:
• smart
• creative
• willing to do great things with great people
• happen to know JS/Flex/Rails
Send whatever can prove your ability to:
© 2010 Idapted, Ltd.
Thank you!
Q&A
http://developer.idapted.com
[email protected] (@jpalley)
[email protected] (@fiyuer)