nubomedia webinar
TRANSCRIPT
Table of contents
1. Introduction2. NUBOMEDIA development3. NUBOMEDIA PaaS4. Let's get to work5. Summary
Table of contents
1. Introduction− Review− What is NUBOMEDIA?− References
2. NUBOMEDIA development3. NUBOMEDIA PaaS4. Let's get to work5. Summary
1. IntroductionReview−Kurento is a media server and a set of APIs aimed to create applications with advance media capabilities−The building blocks for applications are named media elements, chained into media pipelines
1. IntroductionWhat is NUBOMEDIA?−NUBOMEDIA is an open source PaaS (Platform as a Service) based on Kurento−NUBOMEDIA exposes to developers the ability of deploying and leveraging applications with media capabilities
1. IntroductionReferences−Home page
http://www.nubomedia.eu/−Developers guide
http://nubomedia.readthedocs.io/−GitHub organization
https://github.com/nubomedia/−Support for developers
https://groups.google.com/forum/#!forum/nubomedia-dev
Table of contents
1. Introduction2. NUBOMEDIA development
− Architecture− Media API− Kurento vs NUBOMEDIA apps
3. NUBOMEDIA PaaS4. Let's get to work5. Summary
2. NUBOMEDIA developmentArchitecture−The Architecture in NUBOMEDIA is the same than in Kurento
• Three-tier model (inspired in the Web)
Client Application Server Media Server
NUBOMEDIA PaaS
2. NUBOMEDIA developmentArchitecture−Like every application with media capabilities, it is important to distinguish between the media and signaling plane
Client Application Server Media Server
NUBOMEDIA PaaS
signaling
media
2. NUBOMEDIA developmentMedia API−Kurento Media API allows to Java developers consume the media services provided by Kurento Media Server (KMS)−Concepts:
• Media Element• Media Pipeline
2. NUBOMEDIA developmentMedia API−KMS instances are provided elastically by NUBOMEDIA
• The number of available KMS instances depends on the PaaS Manager configuration
−Each KMS has a total amount of available points to create Media Pipelines and Media Elements
• The total points depends on the number of VCPUs of the KMS• The type of the instance can be selected on the PaaS Manager
configuration
Instance type # VCPUs KMS pointsMedium 2 200
Large 4 400
2. NUBOMEDIA developmentKurento vs NUBOMEDIA apps1.Maven dependencies <dependency>
<groupId>org.kurento</groupId> <artifactId>kurento-client</artifactId></dependency>
<dependency> <groupId>de.fhg.fokus.nubomedia</groupId> <artifactId>nubomedia-media-client</artifactId></dependency>
<dependency> <groupId>org.kurento</groupId> <artifactId>kurento-client</artifactId></dependency>
In Kurento apps, we need the kurento-client library to handle KMS from the Java logic
In NUBOMEDIA apps, we need to include also the nubomedia-media-client to discover KMSs within the NUBOMEDIA PaaS
2. NUBOMEDIA developmentKurento vs NUBOMEDIA apps2.Instance of KurentoClient
int sessionPoints = 25; // Chose a convenient value for your pipelineProperties properties = new Properties();properties.add("loadPoints", sessionPoints);KurentoClient kurentoClient = KurentoClient.create(properties);
@Beanpublic KurentoClient kurentoClient() { return KurentoClient.create();}
@Autowiredprivate KurentoClient kurento;
In Kurento, KMS is controlled by an single instance of KurentoClient (for example a Spring Bean)
In NUBOMEDIA, each new media session should create an instance of KurentoClient. The points mechanism is used to locate or create KMS instances (scaling in/out)
2. NUBOMEDIA developmentKurento vs NUBOMEDIA apps3.Deployment: Dockerfile
In Kurento, no Dokerfile is required (deployment is managed by hand)
In NUBOMEDIA, a Dokerfile file is required
FROM nubomedia/apps-baseimage:src
MAINTAINER Nubomedia
ADD . /home/nubomedia
ENTRYPOINT cd /home/nubomedia && mvn spring-boot:run
Table of contents
1. Introduction2. NUBOMEDIA development3. NUBOMEDIA PaaS
− Introduction− PaaS GUI
4. Let's get to work5. Summary
3. NUBOMEDIA PaaSIntroduction−The NUBOMEDIA PaaS manager is a tool aimed to control the way in which the NUBOMEDIA applications are built and deployed inside the NUBOMEDIA PaaS−The capabilities provided by the Paas Manager can be used by developers using the PaaS GUI:
• The PaaS Manager GUI is a web application that allows to use the NUBOMEDIA PaaS Manager
3. NUBOMEDIA PaaSIntroduction−Internally, the NUBOMEDIA PaaS uses Docker containers to deploy applications−Therefore it is a requirement to include a Dockerfile in GitHub repository to be deployed on NUBOMEDIA−Example:
FROM nubomedia/apps-baseimage:src
MAINTAINER Nubomedia
ADD . /home/nubomedia
ENTRYPOINT cd /home/nubomedia && mvn spring-boot:run
https://docs.docker.com/engine/reference/builder/
PaaS GUI−Web application to manage NUBOMEDIA applications
3. NUBOMEDIA PaaS
http://paas-manager.nubomedia.eu:8081/
Credentials needed to login
3. NUBOMEDIA PaaSPaaS GUI−A NUBOMEDIA application can be deployed using the PaaS GUI−It is done providing the GitHub repository URL and a set of configuration parameters
PaaS GUI−Most important configuration values:
3. NUBOMEDIA PaaS
KMS host type:-Medium = 2 VCPUs (200 points)-Large = 4 VCPUs (400 points)
Number of KMSs
GitHub URL repository
Table of contents
1. Introduction2. NUBOMEDIA development3. NUBOMEDIA PaaS4. Let's get to work
− Introduction− Server-side− Client-side− Deployment
5. Summary
4. Let's get to workIntroduction−Devil is in the details, and so, we are going to see a complete NUBOMEDIA step by step−This application is the nubomedia-magic-mirror tutorial
https://github.com/nubomedia/nubomedia-magic-mirror/
http://nubomedia.readthedocs.io/en/latest/tutorial/nubomedia-magic-mirror/
4. Let's get to workServer-side−We recommend Spring-Boot as the base technology for NUBOMEDIA applications
• Spring-Boot embeds a Tomcat server in a simple seamless way for developers
• The application is packaged as a runnable JAR, and when this JAR is executed, the Tomcat server is started and the web application automatically deployed
−We recommend Maven as for managing the life-cycle and managing the dependencies of our applications
4. Let's get to workServer-side−The first step is to clone GitHub repository
−We recommend use an IDE to develop applications. For example, Eclipse (importing app as Maven project)
git clone https://github.com/nubomedia/nubomedia-magic-mirror
4. Let's get to workServer-side−First step is understand the pom.xml file:
<parent> <groupId>org.kurento</groupId> <artifactId>kurento-parent-pom</artifactId> <version>6.6.0</version></parent>
<plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin></plugins>
Inherit from kurento-parent-pom makes simpler our pom.xml by reusing versions defined in the parent
We can use the spring-boot Maven plugin to simplify the packaging as executable JAR
4. Let's get to workServer-side
<dependencies> <!-- Spring --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-websocket</artifactId> </dependency>
<!-- Kurento --> <dependency> <groupId>org.kurento</groupId> <artifactId>kurento-client</artifactId> </dependency> <dependency> <groupId>org.kurento</groupId> <artifactId>kurento-utils-js</artifactId> </dependency>
<!-- Nubomedia --> <dependency> <groupId>de.fhg.fokus.nubomedia</groupId> <artifactId>nubomedia-media-client</artifactId> <version>${nubomedia-media-client.version}</version> </dependency>
…
…
Dependencies clause is one of the most important. Notice that our app depends on Spring, Kurento, NUBOMEDIA and WebJars (for the client-side)
…
4. Let's get to workServer-side−We need to define also the way in which signaling is going to work in our application−For the shake of simplicity, we use a raw WebSocket between the server and client side−We need to define the messages exchanged to carry out the communication about client and server
clientserver
start
startResponse
iceCandidate
onIceCandidate
stop
error
notEnoughResources
Starts a media session
ICE candidates (WebRTC negotiation)
Stops a media session
Error cases
4. Let's get to workServer-side−The class diagram of our application is the following:
MagicMirrorApp
MagicMirrorHandler
UserSession
Main class
Message handler (signaling)
User session (session id, media logic)
4. Let's get to workServer-side
@SpringBootApplication@EnableWebSocketpublic class MagicMirrorApp implements WebSocketConfigurer {
@Bean public MagicMirrorHandler handler() { return new MagicMirrorHandler(); }
@Override public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) { registry.addHandler(handler(), "/magicmirror"); }
public static void main(String[] args) throws Exception { new SpringApplication(MagicMirrorApp.class).run(args); }}
MagicMirrorApp defines the Spring-Boot application and the WebSocket used for signaling ("/magicmirror", managed by MagicMirrorHandler)
4. Let's get to workServer-sidepublic class MagicMirrorHandler extends TextWebSocketHandler {
private final ConcurrentHashMap<String, UserSession> users = new ConcurrentHashMap<>();
@Override public void handleTextMessage(WebSocketSession session, TextMessage message) throws Exception { JsonObject jsonMessage = new GsonBuilder().create().fromJson(message.getPayload(), JsonObject.class); switch (jsonMessage.get("id").getAsString()) { case "start": start(session, jsonMessage); break; case "stop": release(session); break; case "onIceCandidate": onIceCandidate(session, jsonMessage); break; default: error(session, "Invalid message with id " + jsonMessage.get("id").getAsString()); break; } }
MagicMirrorHandler manages signaling messages. It keeps a collection of user sessions (thread-safe map users)
4. Let's get to workServer-sidepublic class UserSession {
public String startSession(final WebSocketSession session, String sdpOffer) { // One KurentoClient instance per session (reserving points per session) Properties properties = new Properties(); properties.add("loadPoints", POINTS_PER_SESSION); kurentoClient = KurentoClient.create(properties);
// Media logic (pipeline and media elements connectivity) mediaPipeline = kurentoClient.createMediaPipeline(); webRtcEndpoint = new WebRtcEndpoint.Builder(mediaPipeline).build(); FaceOverlayFilter faceOverlayFilter = new FaceOverlayFilter.Builder(mediaPipeline).build(); faceOverlayFilter.setOverlayedImage("http://files.kurento.org/img/mario-wings.png", -0.35F, -1.2F, 1.6F, 1.6F); webRtcEndpoint.connect(faceOverlayFilter); faceOverlayFilter.connect(webRtcEndpoint);
// WebRTC negotiation String sdpAnswer = webRtcEndpoint.processOffer(sdpOffer); webRtcEndpoint.gatherCandidates();
return sdpAnswer; }
UserSession handles media logic (media pipeline, media elements, WebRTC negotiation…)
4. Let's get to workServer-side−As of Chrome 47, access to user media can only be done by secure apps (HTTPS), and so, it is needed a Java keystore
server.port: 8443server.ssl.key-store: classpath:keystore.jksserver.ssl.key-store-password: kurentoserver.ssl.keyStoreType: JKSserver.ssl.keyAlias: kurento-selfsigned
application.properties
http://doc-kurento.readthedocs.io/en/stable/mastering/securing-kurento-applications.html
4. Let's get to workClient-side <!-- WebJars -->
<dependency> <groupId>org.webjars</groupId> <artifactId>webjars-locator</artifactId> </dependency> <dependency> <groupId>org.webjars.bower</groupId> <artifactId>demo-console</artifactId> </dependency> <dependency> <groupId>org.webjars.bower</groupId> <artifactId>adapter.js</artifactId> </dependency> <dependency> <groupId>org.webjars.bower</groupId> <artifactId>jquery</artifactId> </dependency> <dependency> <groupId>org.webjars.bower</groupId> <artifactId>ekko-lightbox</artifactId> </dependency></dependencies>
…Client-side dependencies are handled by Maven
WebJars allows to declare client-side dependencies (i.e. CSS and JavaScript libraries) also in the pom.xml. These libraries are used linked in the HTML client-side pages
4. Let's get to workClient-side−The app has been implemented as a SPA (single page application) which contains a single HTML
<!DOCTYPE html><html><head><link rel="stylesheet" href="webjars/bootstrap/dist/css/bootstrap.min.css"><link rel="stylesheet" href="webjars/ekko-lightbox/dist/ekko-lightbox.min.css"><link rel="stylesheet" href="webjars/demo-console/index.css"><link rel="stylesheet" href="css/styles.css">
<script src="webjars/jquery/dist/jquery.min.js"></script><script src="webjars/bootstrap/dist/js/bootstrap.min.js"></script><script src="webjars/ekko-lightbox/dist/ekko-lightbox.min.js"></script><script src="webjars/adapter.js/adapter.js"></script><script src="webjars/demo-console/index.js"></script><script src="js/kurento-utils.js"></script><script src="js/index.js"></script><title>NUBOMEDIA Tutorial: WebRTC Magic Mirror</title></head>
JavaScript/CSS are linked here. Using Bootstrap to provide responsive design
4. Let's get to workClient-side−A JavaScript file contains the client-side logic (message handling and WebRtcPeer use)
var ws = new WebSocket('wss://' + location.host + '/magicmirror');
ws.onmessage = function(message) { var parsedMessage = JSON.parse(message.data); switch (parsedMessage.id) { case 'startResponse': startResponse(parsedMessage); break; case 'error': onError("Error message from server: " + parsedMessage.message); stop(false); break; case 'iceCandidate': webRtcPeer.addIceCandidate(parsedMessage.candidate, function(error) { if (error) return console.error("Error adding candidate: " + error); }); break; // ... }}
4. Let's get to workClient-side−A JavaScript file contains the client-side logic (message handling and WebRtcPeer use)
var webRtcPeer;
function start() { var options = { localVideo : videoInput, remoteVideo : videoOutput, onicecandidate : onIceCandidate } webRtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options, function(error) { if (error) { return console.error(error); } webRtcPeer.generateOffer(onOffer); });}
4. Let's get to workDeployment−We need a Dockerfile to deploy our app
FROM nubomedia/apps-baseimage:src
MAINTAINER Nubomedia
ADD . /home/nubomedia
ENTRYPOINT cd /home/nubomedia && mvn exec:java
4. Let's get to workDeployment−Finally we can deploy our app in the NUBOMEDIA PaaS:
4. Let's get to workDeployment−The app changes its state, from CREATED to RUNNING (it takes a couple of minutes to finish):
4. Let's get to workDeployment−We can trace two types of logs: build and app
I0708 12:51:02.335080 1 builder.go:41] $BUILD env var is {"kind":"Build","apiVersion":"v1","metadata":{...} I0708 12:51:02.423366 1 source.go:96] git ls-remote https://github.com/nubomedia/nubomedia-magic-mirror --headsI0708 12:51:02.423547 1 repository.go:275] Executing git ls-remote https://github.com/nubomedia/nubomedia-magic-mirror --headsI0708 12:51:04.931566 1 source.go:189] Cloning source from https://github.com/nubomedia/nubomedia-magic-mirror
...
Image already existslatest: digest: sha256:24ac51ca448edfb1582f0bffc990c95506065f0aa26d5b295d1cb32d35ce2dfe size: 49623I0708 12:54:31.392578 1 docker.go:99] Push successful
4. Let's get to workDeployment−We can trace two types of logs: build and app[INFO] Scanning for projects...[INFO] [INFO] ----------------------------------------[INFO] Building NUBOMEDIA Java Tutorial - Magic Mirror 6.5.0[INFO] ----------------------------------------[INFO] [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ nubomedia-magic-mirror --- `.-:////:-.` -+shmmmmmmmmmmmmmmhs/- `/ymmmmmmmmmmmmmmmmmmmmmmmms/` -smmmmmmmmyo+/:----:/+ohmmmmmmmms. -ymmmmmmy+. -+hmmmmmmy. -------` `ommmmmd+` .+dmmmmmo +oooooo- .hmmmmm+ `ommmmmh` +oooooo- .dmmmmh. ``````` +oooooo- sdddddds `dmmmmy .dhhhhhh. ```````` smmmmmmy ./+osyysymmmmy .mmmmmmm. smmmmmmy `:sdmmmmmmmmmmmmd` .mmmmmmm. -----` +yyyyyy+ `+dmmmmmmdhyyyhdmm+ .hhhhhhh. ooooo- -------- -dmmmmmy:` `:` ooooo- .----. oooooooo` -mmmmmy. .....` hmmmms oooooooo` dmmmms yhd- hmmmms oooooooo` :mmmmm` syy- /++++: :::::::: /mmmmm `mmmmm/ -ss: os` os` -s/ -ssssss+- .+syys+. -//. ://` .::///- -/////:. // .//` +mmmmm: :mdm/ hm. hm. /mo :my---/mm`:md:..:dm/ /o+o` -o+o` /o:.```` :o:``.:o+ oo `o/++ smmmmh :ms:m+ ym. hm. /mo :mh+++smo ym/ :mh /o.o: `o/:o`.oo:::::. :o- /o- oo +o`.o/ /mmmh :ms :moym. hm. /mo :my:::+mh sm+ /my /o.-o./o`/o`.oo.....` :o- +o- oo /o+//+o: `odh :ms -mmm. +my::/dm- :my///omm`.dm+::+mm- /o. +oo: /o` :o/----. :o/---/o/ oo -o:..../o. - .+: .++` ./+++:` .++++++:` `:+oo+:` .-` `-- .-` `-----. .------` -- -- `--
...2016-07-08 12:55:32.702 INFO 6 --- [irrorApp.main()] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8443 (https)2016-07-08 12:55:32.713 INFO 6 --- [irrorApp.main()] e.n.tutorial.magicmirror.MagicMirrorApp : Started MagicMirrorApp in 10.529 seconds (JVM running for 31.135)
Deployment−Ta da!
4. Let's get to work
Table of contents
1. Introduction2. NUBOMEDIA development3. NUBOMEDIA PaaS4. Let's get to work5. Summary
− Things to remember for NUBOMEDIA apps
5. SummaryThings to remember for NUBOMEDIA apps−From a high level point of view:
• The media-api concepts (media pipelines, media elements) are exactly the same in NUBOMEDIA
• The NUBOMEDIA PaaS is going to provide elastically instances of KMSs to our application
−From a low level point of view:1. We need to add the nubomedia-media-client dependency2. We need to create a KurentoClient instance per media
session in order to inform the PaaS when KMSs are required3. We need a Dockerfile to configure the execution point of our
application