Aalto University ELEC-E8004 Project work course Year 2019

Project plan

Project #16 Preparing simulation models for robotic

tasks

Date: 04.02.2019

Calvin Guillot Suarez Eetu Savolainen Juuso Leskinen

Mikko Siltala Orgilbold Zardikan

Petri Kalima

1

Information page Students Calvin Guillot Suarez Eetu Savolainen Juuso Leskinen Mikko Siltala Orgilbold Zardikan Petri Kalima Project manager Juuso Leskinen Official Instructor Karol Arndt Starting date 10.1.2019 Approval The Instructor has accepted the final version of this document Date: 04.02.2019

2

1) Background In modern manufacturing, robotic solutions offer improvements in efficiency and safety. Over time, the need for robotics in other environments might also build up. In the future, one might run into robotic solutions in everyday situations, for example when entering buildings and being welcomed by a robot who politely opens the door for them. These scenarios might still feel like a distant thought. However, developing robotic solutions for variety of tasks has its benefits. If not in practical use, then at least in research and teaching. The development of robotic tasks often involves simulation. Simulation provides a safe and inexpensive environment for developing and testing some physical system, including a robot. It is safe because mistakes made during the development can be detected from the simulation before the developed solution is applied to a real system. It is cheaper because working and testing in simulation is less time consuming than using a physical system. In addition, when using methods such as reinforcement learning which requires large number of trials, simulation is definitely preferable over the real system.

2) Expected output The aim of this project is to develop MuJoCo simulation models for a door opening task using the Franka Panda robot (Figure 1). The end result will include the door’s 3D and dynamical models integrated into the MuJoCo simulation environment, where the door is manipulated by the robot.

Figure 1: Franka Panda robot (Franka presskit [31.1.2019] Available at:

https://www.franka.de/presskit) The model is to be a tool for the Intelligent Robotics research group at Aalto University, that does research in robotics, computer vision and machine learning. The results of our project are planned to be used in sim-to-real experiments in machine learning, where reinforcement

3

learning would be used to teach the robot. The research group wants this model because there are complicated contact forces present in the manipulation task of interacting with a door handle and a door on hinges. The end users are expected to be members of the research group. The model should be parameterized to allow training on varying sizes and shapes of doors, and the simulation should also have a Python interface for easy access and experimentation. The resulting models and code should also be of sufficient quality so that the end user can easily make any necessary modifications to them. The functionality of the resulting models will be demonstrated in the simulation with the Franka Panda robot. In the demonstration the robot is expected to be able to manipulate the door handle and to be able to open and close the door in the simulation environment.

3) Phases of project The table below shows the planned phases for the project and their respective milestones.

Milestone Description Timeline and deadline

M1 Project plan 10.1.19 - 4.2.19

M2 Door model 28.1.19 - 22.2.19

M3 Business seminar slides 26.2.19 - 8.3.19

M4 Presentation and Business aspects seminar 26.2.19 - 12.3.19

M5 Business aspects document 11.3.19 - 15.3.19

M6 Integration 18.3.19 - 29.3.19

M7 ROS code 4.2.19 - 26.4.19

M8 Python interface 29.4.19 - 3.5.19

M9 Poster 6.5.19 - 13.5.19

M10 Final presentation/demo and Final Gala 6.5.19 - 21.5.19

M11 Final report 21.5.19 - 31.5.19

4

4) Work breakdown structure (WBS) The work breakdown was approached in the form of functional, system and software breakdowns. These breakdowns iteratively approach an easier and more tangible breakdown of the work. The iteration process is presented in a more visually appealing way in Figure 2, and then in more detail as a list. This breakdown omits all of the work required for the project planning, business aspects, final reporting and learning of required software.

Figure 2. Work breakdown and internal linkage

1. Functional breakdown

1.1. Simulation 1.1.1. Door model 1.1.2. Robot model

1.2. Interface 1.2.1. ROS 1.2.2. Python

2. System breakdown 2.1. Door model

2.1.1. 3D CAD model 2.1.2. Dynamic model

2.2. Robot model 2.2.1. Integration

2.3. ROS 2.3.1. Robot commanding package 2.3.2. Changeable simulation parameters

2.4. Python 2.4.1. Interface to command robot

5

2.4.2. Interface to change simulation parameters 3. Software breakdown

3.1. CAD model 3.1.1. Measure dimensions 3.1.2. Create a model

3.2. Dynamic model 3.2.1. Measure forces 3.2.2. Create XML-files

3.3. Integration 3.3.1. Bring the models together

3.4. ROS package for commanding robots 3.4.1. Grasping the handle 3.4.2. Turning the handle 3.4.3. Opening/closing the door

3.5. Changeable simulation parameters 3.5.1. Door dynamics 3.5.2. Door dimensions

3.6. Interface to command robot 3.6.1. Python interface for the ROS package

3.7. Interface to change simulation parameters 3.7.1. Python interface for the ROS simulation parameters

5) Work packages and Tasks of the project and Schedule

5.1-5.2) Work packages and Tasks The work packages (WP) were divided into tasks, listed in the table below, with their work package leader and estimated working hours set aside for it. Table also shows related milestones for each work package. The role of a WP leader is described in more detail later in the document. Estimated working hours for each task are presented in the next section.

Work Package WP and Tasks Team and WP leader Working hours

Milestones

WP1 Project plan 1.1.Write the project plan

Calvin, Eetu, Juuso, Mikko, Orgilbold, Petri

130h M1

WP2 Modeling 2.1.Measure the door 2.2.CAD modeling 2.3.Dynamic modeling

Calvin, Eetu, Petri 350h M2

6

WP3 Business plan 3.1.Prepare presentation for the

Business aspects seminar 3.2.Create presentation slides 3.3.Write the Business aspects

document

Calvin, Eetu, Juuso, Mikko, Orgilbold, Petri

150h M3, M4, M5

WP4 Integration 4.1.Prepare a launch file that has the

dynamic model and the robot model

Calvin, Eetu, Juuso, Mikko, Orgilbold, Petri

100h M6

WP5 ROS simulation 5.1.Familiarise with ROS 5.2.Divide ROS coding into tasks 5.3.Prepare ROS simulation

Juuso, Mikko, Orgilbold

350h M7

WP6 Python interface 6.1.Create Python interface for the

simulation

Eetu, Petri 20h M8

WP7 Finalize the project 7.1.Create a poster 7.2.Create a demo and prepare a

presentation for the Final Gala 7.3.Write the final report

Calvin, Eetu, Juuso, Mikko, Orgilbold, Petri

250h M9, M10, M11

5.3) Detailed schedule Below is the overview of the work packages in a Gantt chart:

7

And a detailed schedule with estimated working hours:

6) Work resources

6.1) Personal availability during the project The table below shows the number of hours available to be spent on the project per person per week, excluding lectures and seminars.

8

Calvin Eetu Juuso Mikko Orgilbold Petri Info

Week 2 2 2 2 12 2 2

Week 3 2 2 6 12 6 6

Week 4 0 12 12 12 5 12

Week 5 10 12 12 12 5 16

Week 6 12 12 12 6 5 12 Project plan deadline

Week 7 10 10 11 12 10 10

Week 8 12 12 12 12 10 12 Exam week, Business aspects

Week 9 12 12 12 8 10 12 Business aspects

Week 10 12 12 0 8 12 12 Business aspects

Week 11 0 12 12 8 10 12 Business seminar

Week 12 12 12 12 8 10 10

Week 13 14 12 10 8 10 10

Week 14 14 6 0 0 7 6

Week 15 14 12 12 8 15 0 Exam week

Week 16 12 12 16 16 18 10

Week 17 12 12 16 16 18 10

Week 18 14 12 16 0 20 12

Week 19 14 12 12 16 15 16

Week 20 14 12 16 16 15 16

Week 21 14 10 12 16 12 12 Final Gala

Week 22 14 10 12 16 10 12 Final deadline, Exam week

Total 220 220 225 222 225 220 Total: 1332

9

6.2) Personal goals Calvin: My main objective is to improve my skills in programming and modeling. I want to excel in creating an accurate model of the problem, physically and mathematically. Similarly, I want to be able to work better in a diverse team as this one. I think it is important to give yourself responsibilities and stick to them. I would also get to know ROS and MuJoCo solutions, given that I have never used them before. I know that regardless of my involvement, I will learn different skills that I could use in the future. Eetu: I want to improve my programming skills and become familiar with ROS and MuJoCo. I also want to get more experience working on a project as a part of a group. Juuso: My primary goal is to finish this project and produce an outcome that really benefits the research group. I want get used to Linux and Linux terminal, and learn ROS and MuJoCo. Working in a project this size is also interesting and new for me. In addition, I was chosen as the project manager and I hope I can fill this role. I want to be a better leader someday and hopefully this will prepare me for that. Mikko: With this project my goal is to improve my programming skills in C++ and become more proficient in using ROS and MuJoCo. Thus far I’ve been coding mostly for course exercises, and I have little experience in coding for a customer or as a part of a project. Therefore I am looking forward to working in the ROS part of this project. My other goal is to gain experience in working on larger scale projects, which require planning and management to be successful. Orgilbold: I would like to become a competent user of ROS and MuJoCo and working with robots. I am also excited to learn a new skill not just to pass an exam but to actually make a project happen. Throughout my university career I think I have learnt well how to learn for exams but I am afraid that my skills are not retained in the long term. Petri: I am hoping to improve my software development skills in Python, especially regarding simulations and working with robotics. Large-scale projects tend to be great for learning and I think it’s interesting to work with a project that is not purely software but also contains things like modeling. I’m hoping that this project can show me what kind of tasks can I expect when working on software related issues for robotics.

7) Cost plan and materials There is no budget allocated for this project, and the predicted expenses are zero. Furthermore, no member of the group is to make purchases on their own expense. In the

10

unlikely scenario where unpredicted and unavoidable expenses arise the group will be in contact with the instructor and course management. This project is completed in software, and does not produce any new hardware, and therefore won’t be needing any material/component e.g. purchases. The costs associated with a software project would traditionally be associated to getting access to necessary software and processing power. Most of the students however already have access to the required software they will be using via free student versions, and the remaining students can also receive the student licenses. The resources allocated to this project are further described in chapter 8.

8) Other resources As the project is a software project, the amount of other required resources is minimal. All the software needed for the project can be used on regular computer hardware, as they are not very computationally demanding. Therefore, any specific computers are not needed, and the students can run them on their personal computers. Some equipment will be needed for the measurements used for the model, such as the dimensions of the door and the required forces. The required instruments, such as a tape measure and scale, will be provided for the students and can be found in the same lab where the measured door is located. None of the tools will require official instruction before use.

9) Project management and responsibilities This chapter tries to state clearly the responsibilities of each role. Project manager is overall responsible for the project. He communicates with the team, supervisor and course staff and informs the team members. He tries to have an understanding of the overall progress and makes sure everything gets done by the deadline. He is prepared for each meeting, for example prepares questions or topics for discussion and informs them to the team beforehand. He should also mention if there are some material that team members should inspect/prepare for the meeting. Finally, project manager also does his own part of the project like everyone else. Instructor is both the teacher and the supervisor. He is ready to help when asked, gives advice but does not complete the task in behalf of the team. In the early stage of the project, the instructor helps the team to understand the nature of the project, for example what parts

11

the project consists of and how much time would be needed for each part. At the end, the instructor evaluates the outcome of the project. Work package leader has similar responsibilities to project manager but in the scale of that particular work package. He communicates with the project manager, instructor and the team members assigned to that work package. WP leader is responsible for that work package. Team member is assigned with certain tasks and he is responsible for completing them. Team member’s main focus should be in doing and less in management and communication. He primarily communicates inside the work package team. However, because of the size of this project, team member communicates often with the whole team. Other advisors would provide external support in case it’s necessary. They would help solve problems or guide the team when asked. At least for now, we do not have other advisors helping us.

10) Project Meetings It was decided that all team members and the Instructor would have a regular weekly meeting with a standard duration of 1 hour for regular progress sharing and discussion. The meetings can be longer, for example for the project planning phase. For the first few meetings we decided to alternate the role of the memokeeper, so everyone has a chance to experience it and learn from it, afterwards the memokeeper is mainly going to be Orgilbold, unless otherwise agreed. The memos will be in our shared Google Drive folder for easy access and sharing. This also allows others members to write down some additional notes that the memokeeper did not write down but others thought it was useful. Our WhatsApp communication channel is used to alert others about potential unavailability for the regular meetings or requesting additional ones.

11) Communication plan We have agreed to use a Slack-workspace as a communication method since it is generally accepted as faster and more convenient than email, while still providing a better way to separate and keep track of different conversations and threads than WhatsApp. We do also have a WhatsApp-group which is commonly used for things such as discussing the location of the next meeting. Each member of the group is part of the Slack-workspace and the WhatsApp-group, including the instructor. So far the communication has been swift and breakdowns have not happened. Slack covers most of our communication, but email has also been used for certain issues which did not require immediate responses. For software-related bugs, Gitlab issue trackers or similar may be used.

12

Face-to-face communication might happen not only in meetings as mentioned, but also during the lectures where we will see each other quite often. Indirect communication might happen in the form of software that offers version control. We use Aalto Version for software and Google Docs for documents, which both offer ways to see the changes made by other group members. This allows us to see what kind of progress has been made without having to ask about it separately through messages.

12) Risks We have built a cohesive project plan, however there are some risks that are worth taking into consideration. The following table presents the associated risks that might arise during the project. The risk level is defined by the likelihood of an event against the consequences of said event. Within each item, there is a recommended action that could be implemented to mitigate the risk.

RISKS

Likelihood Severity Description Actions

Rare

Medium Tools are not working properly Follow the recommended use of software

High Data loss Backup regularly

Unlikely

Low Taking too many measurements Try to simplify the measurements

Medium Poor communication Be open about issues and communicate regularly

High Team member unable to continue with the project

Regroup based on the WP organization

Possible

Medium Team member unable to attend meetings or finish his tasks

Try to meet the schedules and report in advance

Medium Unreadable code Use version control consistently

High Project delays Try to stick to the project plan and schedule spare time for unexpected events

13

13) Quality plan Each Work Package (WP) has a main person responsible for it. This person is in charge of a team and he has to make sure the tasks are progressing along. The project manager will regularly check our progress with the schedule. The Instructor could act as a beta tester to see if our simulation meets the research group’s needs. The following table describes the arrangement of the sub-teams per each WP.

Work Package WP# Leader Team

WP1 Project plan Juuso Calvin, Eetu, Juuso, Mikko, Orgilbold, Petri

WP2 Modeling Petri Calvin, Eetu, Petri

WP3 Business plan Orgilbold Calvin, Eetu, Juuso, Mikko, Orgilbold, Petri

WP4 Integration Calvin Calvin, Eetu, Juuso, Mikko, Orgilbold, Petri

WP5 ROS simulation Mikko Juuso, Mikko, Orgilbold

WP6 Python interface Eetu Eetu, Petri

WP7 Finalize the project Juuso Calvin, Eetu, Juuso, Mikko, Orgilbold, Petri

Furthermore given the nature of the tasks, each team member check the quality of different WP. This guarantees that there are at least 2 people working in a WP at all times. To ensure a streamlined quality control, each person is responsible of reporting bugs, drawbacks, errors, or delays that might happen during the development at any stage of any WP. These reports will be done through the WP channels on Slack. Each WP leader will be responsible to check the quality of the tasks against the expected requirements of the project, for each iteration of said tasks. A WhatsApp channel will be used for urgent matters. We will use version control for the code and revise each released build. Then we can assess bugs and other errors that might appear during the production. For example, the simulation will have several iterations, and each iteration will be checked by the person responsible of that work package (WP5).

14

14) Changing this plan Anyone in the group can make an initiative to ask for changes that they would like to see. If anything like this comes up, the changes would preferably be mentioned in meetings or by messaging others before they are made. Changes should be discussed by at least the people who are involved with that particular issue. Ideally if someone is not present, we can discuss the proposed changes with them through Slack/email. Changes regarding the schedule should be arranged so that other tasks aren’t at the risk of missing a deadline. The project plan is being worked on with Google Docs, which supports version control. Therefore if a certain change turns out to be worse than expected, the project plan can be changed back to the original rather easily.

15) Measures for successful project In the end, it can be determined if the project was successful from three different perspectives: (1) The outcome of the project, (2) project progression and (3) personal learning goals. Final outcome of the project is the simulation model. Benefit of simulation is that it can be tested simply by running the simulation. In addition, in case there is enough time left at the end, testing the model on the real robot would be the best evaluation method. It would help to determine the quality of two main aspects: the accuracy of the dynamic model and the simulation. Code readability should also be noticed. Evaluating how the project is progressing can be done by documenting when the scheduled tasks are completed and when the milestones are reached and comparing them to the initial plan. This can be beneficial in the future if we see ourselves under- or overestimating the resources needed for each task. In addition, it is important to document not only the time used but also by the quality of the deliverables. By going through the deliverables with the team and our instructor we can determine the quality of our work. We can also do comparison with some of the previous year’s project documentations or ask someone outside our group to evaluate our work. Google Docs document will be created for the documentation of the project progression. When a task/milestone/WP is finished, we will write down how long it took (timeline and working hours) and try to determine its quality and explain it in the document. Personal learning is evaluated best by the person himself. In the end, it is good to look back at the personal learning goals and think if they were achieved.

15