3D Software Techniques for Surgery

2017.10.18

한국과학기술연구원 의공학연구소

김 영 준

3D 의료영상처리 S/W

2

▶ 3D 가상 수술 :시술방법 최적화수술가이드 설계 및 3D 프린팅 지원간편한 수술 계획 인터페이스효과적인 진단 정보 가시화

▶ 3D 환자모델링 :신경, 혈관, 뼈, 피부 등 복합가시화환자 모델링 자동화

▶ 3D 형상 모델 처리기술 :Mesh HealingBoolean Operation3D Sketch

3D 환자모델링 – Shoulder MR

3

3D 회전근개 모델링

정상인 대파열 환자 소파열 환자

회전근개 모델 3차원 뷰어

: Volume data + STL mesh models 복합

Segmentation Results (Supraspinatus)

3차원 모델링의 실험 및 평가- 실험 데이터 수 : 5개- 비교 방법 수 : 5가지 (기존 영상분할 방법 2가지(RSF, RE), 수동 분할, 전문가1,2에 의한 각 분할)

- 분할 정확도 측정: Dice Ratio(DR) =2|𝐴⋂𝐵|

𝐴 +|𝐵|, 𝑨 : 참조분할의모든복셀, 𝐵 : 실험 결과의모든복셀 → 1에 가까울수록실험 결과를신뢰할수있음.

파란색: 전문가1,2에 의한분할 결과들간의 dice ratio

빨간색: 전문가1과각 방법간의 dice ratio

연두색: 전문가2와각 방법간의 dice ratio

→ 다른방법들에비해전문가분할결과간의 dice ratio와유사한범주를가지며, 0.94 이상의높은신뢰도를 가짐

3차원 모델링 오차 2차원 영상간의 비교

RSF RE원 영상 제안된방법 전문가1 전문가2

3.0mm

0mm

RSF RE 수동 분할 제안된방법

3.0mm

0mm

RSF RE 수동 분할 제안된방법

5.13 mm 0.95 mm 0.86 mm 0.44 mm 평균오차

2.53 mm 0.75 mm 0.65 mm 0.52 mm 평균오차

3D 프린팅활용의학실습

5

서울대학교의과대학해부학수업, 2015. 5. 21 / 2016. 6. 3 / 2017. 6. 2

Contents

3D Maxillofacial Reconstructive Surgery

• Mandibular Reconstruction

• Orbital Fracture Reconstruction

6

Patient-specific Surgical Guide using 3D Printing

7

8

3D virtual surgical planning

Automatic 3D patient modeling

FEM analysis for optimization of surgical procedures

Automatic 3D patient modeling (mandible, fibula, vessels, etc.)

Virtual surgery – arrangement of bones, optimization of fibula free flap

FEM analysis – optimization of plates, screw arrangements, surgical procedures,

3D Surgical Planning

9

Preparation Time (>10 hours)

Manual Segmentation

Not specialized in maxillofacial recon. surg.

Expensive, Foreign Company (> 40,000 USD)

MimicsTM (Materialise, Belgium)

Problems of existing S/W Clinical Needs

Specialized S/W for maxillofacial recon. surg. for

Korean patients

Automation of manual process

Easy to use

Optimized virtual surgical planning

Preoperative Virtual Simulation

Prof. Jung-woo Lee (Kyunghee Univ.)

Prof. Jong Woo Choi (Asan Medical Center)

Dr. Yong-chan Lee (Bastian Seoul Hospital)

공동연구: 경희대학교 치과 구강외과 이정우 교수님

3D Surgical Planning

10

Bone Resection

Cutting plane

Boolean Operation

(Subtraction)

11(a) Define of resection area (b) Removal of resection area

3) 3D Surgical Planning (Automatic)

12

Modeling of Surgical Cutting Guide (1)

13

Modeling of Surgical Cutting Guide (2)

14

3D Printing Results

15

Comparison with a Commercial S/W

16

Mimics & 3-matic(Materialize co.)

Developed S/W(KIST)

PlanningAutomatic planning

of mandible reconstruction

X O

CADFunctions

Healing O O

Boolean O O

Translation O O

Rotation O O

Mate Δ O

Multithreading X O

Cutting Plane

Creation O O

Modification Δ O

Thickness Δ O

CT image projection X O

Display in CT slice O O

Contour of bone X O

View

Multi-rendering view

X O

CT slice view Δ O

Sub-view for fibula X O

MeshManagement

Mesh export O O

Color/Transparency Δ O

Transform Lock X O

User Study(Department of Oral and Maxillofacial Surgery,

Kyunghee Univ.)

• User-friendly interface

• Easy to use

- Multi-rendering view, sub-view for fibula

- Transform lock

- Reduced planning time

User Study

17

4.23 4.26 4.41

3.26 3.24 3.22

0

1

2

3

4

5

편의성 유용성 소요시간

사용자설문평가결과

KIST Mimics

대한악안면성형재건외과학회 전문가 설문조사

2016.11.4, 광주 김대중컨벤션센터

[설문 응답자]

• 소속: 이대 목동병원, 서울대 치대 임상시험센터,

단국대, 가톨릭 관동대 국제성모병원, 경희대 치대(2)

• 신분 및 연차: 교수 (2), 전임의 (3), 치과 대학원생

• 수술 경험: 무(2), 1회, 5~7회, 20회, 어시스트 2년

18

Contents

3D Maxillofacial Reconstructive Surgery

• Mandibular Reconstruction

• Orbital Fracture Reconstruction

19

3D Planning S/W for Orbital Fracture Reconstruction

20

AMC-KIST 공동연구 과제

공동 PI: 서울아산병원 성형외과 최종우 교수님

Orbital wall modeling(Semi-automatic segmentation)

Fracture display 3D analysis Patient-specific Implant design

Mirroring & Registration

Better surgical outcomeShortening surgical time in OREnhancing patient safety

21

3D Patient Modeling

Semi-Automatic Orbital Wall Modeling

기존 SW: 100장 이상의 영상에 대해 수작업

모델링 소요시간 1/10 수준으로 감소

Patient #1 Patient # 2 Patient # 3

Facial bone CT Thresholding Ethmoid 분할 Orbital wall 분할 3D model

제안 기술

Design of Patient-specific Plate

3D Diagnosis

23

Measurement (length, angle, volume, ex- or enophthalmos)

Multiple display (orbit, optic nerve)3D orbit volume

Real-time Surgical Navigation (1)

24

Pre-Op. Planning

Patient

CT scanning, 3D modeling

Virtual surgical planning

Patient-specific plate

Registration

Patient 3D virtual model (plate + marker)

Intra-Op. Navigation

Real-time surgical navigation

Real-time Surgical Navigation (2)

25

Anatomical Landmark(외이도관 및 치아)

Portal forlandmark

Upper teeth model

치아부설계

Patient-Specific 광학식 추적 마커 프레임

3D Printed frame

Design SW

ApplicationsPlastic surgery, Orthognathic surgery, Orthopedic surgery, etc.

26

의료기기인증

27

의료기기법에 따른 의료기기 소프트웨어의 품질관리시스템 구축

품목군: 생체현상 측정기기

품목명: 의료영상 분석장치 소프트웨어(A26430.11), 2등급

• 식품의약품안전처 임상 GMP 적합인정서 획득 (2016.11)

임상시험계획승인신청

임상시험계획승인신청서

임상시험계획서

임상시험실시기관승인서 (IRB)

임상 GMP 적합인정서

기술문서 및관련 자료

제조시설/공정도

소프트웨어 품질기록, 데이터

소프트웨어 검증 및 유효성 확인 보고서

품질매뉴얼

절차서

GMP 적합인정신청서

임상 GMP 적합인정서

지침서

제품표준서

성능 및 유효성확인 자료

* GMP(Good Manufacturing Practice) – 의료기기 제조 및 품질관리 기준

정형외과공동연구 (1) 건국대학교/네온정형외과 박진영 교수님

28

(a)

(c)

(b)

(d) (d)

29

정형외과공동연구 (2) 건국대학교 정석원교수님

30

S. Kim, D. Lee, S. Park, K. Oh, S.W. Chung*, Y. Kim*, "Automatic segmentation of supraspinatus from MRI by

internal shape fitting and autocorrection", Computer Methods and Programs in Biomedicine, Vol. 140, pp. 165-

174, Mar. 2017

S.W. Chung, …, Y. Kim*, "Serial Changes in 3-Dimensional Supraspinatus Muscle Volume following Rotator

Cuff Repair", The American Journal of Sports Medicine, Vol. 45, No. 10, pp. 2345-2354, Aug. 2017

31

S.W. Chung, …, Y. Kim*, "Serial Changes in 3-Dimensional Supraspinatus Muscle Volume following Rotator

Cuff Repair", The American Journal of Sports Medicine, Vol. 45, No. 10, pp. 2345-2354, Aug. 2017

정형외과공동연구 (3) 강동경희대학교 김강일 교수님

32

X-ray imagesCT data

3D scan dataRegistration

Youngjun Kim, Kang-Il Kim*, Jin Hyeok Choi, Kunwoo Lee, "Novel methods for 3D postoperative analysis of

total knee arthroplasty using 2D–3D image registration", Clinical Biomechanics, Vol. 26, No. 4, pp. 384-391,

May 2011

Automatic

calculation of

relative angles

External-internal

rotation

Extension &

Flexion gap

정형외과공동연구 (4) 삼성서울병원 왕준호 교수님

한림대학교 이병훈 교수님

33

Arthroscopic Screen

Navigation

Cadaver

Cadaver test

Animal test

Y. Kim, B.H. Lee, K. Mekuria, H. Cho, S. Park, J.H. Wang, D. Lee, "Registration accuracy enhancement of a surgical navigation system for

anterior cruciate ligament reconstruction: A phantom and cadaveric study", The Knee, S0968-0160(16)30248-4, Feb. 2017

B.H. Lee, D.H. Kum, I.J. Rhyu, Y. Kim, H. Cho, J.H. Wang, "Clinical advantages of image-free navigation system using surface-based

registration in anatomical anterior cruciate ligament reconstruction", Knee Surgery, Sports Traumatology, Arthroscopy, Vol. 24, No. 11, pp.

3556-3564, Nov. 2016

J.G. Seo, S.M. Kim, J.M. Shin, Y. Kim, B.H. Lee, "Safety of simultaneous bilateral total knee arthroplasty using an extramedullary referencing

system: results from 2098 consecutive patients", Archives of Orthopaedic and Trauma Surgery, Vol. 136, No. 11, pp. 1615-1621, Nov. 2016

E. Shim, Y. Kim, D. Lee, B.H. Lee, S. Woo, K. Lee, “2D-3D registration for 3D analysis of lower limb alignment in a weight-bearing condition”,

Applied Mathematics (Accepted)

2D-3D registration

34

35

36

E. Shim, Y. Kim, D. Lee, B.H. Lee, S. Woo, K. Lee, “2D-3D registration for 3D analysis of lower limb alignment in a weight-bearing condition”,

Applied Mathematics (Accepted)

ACL 3D 분석

37

Mechanical Axis 3D 분석

38

AI기반 3D 수술 S/W 기술

39

준비단계

환자 3D 데이터

AI 수술진단

Classification

AI 수술계획

치료방법추천

AI 수술분석

수술결과분석

Deep Learning based Rotator Cuff Tear Diagnosis

40

Rotator cuff tear 자동진단 2000 tear & 2000 normal data 학습 후 진단 accuracy > 93%

Deep Learning based Rotator Cuff Tear Diagnosis

41

Book - VTK 프로그래밍

42https://github.com/vtk-book/example 42

Summary

3D 의료 프린팅 S/W 기술

응용개발: 3D 악안면 재건 S/W

정형외과 협력연구 사례

Journal Paper Publication (2016~)1. S.W. Chung, …, Y. Kim, "Serial Changes in 3-Dimensional Supraspinatus Muscle Volume following Rotator Cuff Repair", The American Journal of Sports Medicine,

Vol. 45, No. 10, Aug. 2017

2. Y.D. Choi*, Y. Kim*, E. Park, “Patient-Specific Augmentation Rhinoplasty Using a Three-Dimensional Simulation Program and Three-Dimensional Printing”,

Aesthetic Surgery Journal, DOI: 10.1093/asj/sjx046, May, 2017

3. S. Kim, D. Lee, S. Park, K. Oh, S.W. Chung, Y. Kim, "Automatic segmentation of supraspinatus from MRI by internal shape fitting and autocorrection", Computer

Methods and Programs in Biomedicine, Vol. 140, pp. 165-174, Mar. 2017

4. Y. Kim, B.H. Lee, K. Mekuria, H. Cho, S. Park, J.H. Wang, D. Lee, "Registration accuracy enhancement of a surgical navigation system for anterior cruciate ligament

reconstruction: A phantom and cadaveric study", The Knee, S0968-0160(16)30248-4, Feb. 2017

5. C. Kyu Lee, Y. Kim, N. Lee, B. Kim, D.Y. Kim, S. Yi, "Feasibility study of utilization of action camera, GoPro Hero 4, Google glass and Panasonic HX-A100 in Spine

surgery", Spine, Vol. 42, No. 4, pp. 275-280, Feb. 2017

6. Q.C. Nguyen, Y. Kim, H. Kwon, "Optimization of layout and path planning of surgical robotic system", Int’l Journal of Control, Automation and Systems, Vol. 15, No.

1, pp. 375-384, Jan. 2017

7. Q.C. Nguyen*, Y. Kim*, S. Park, H. Kwon, "End-effector path planning and collision avoidance for robot-assisted surgical system", Int’l Journal of Precision

Engineering and Manufacturing, *These authors contributed equally to this work, Vol. 17, No. 12, Dec. 2016

8. B.H. Lee, D.H. Kum, I.J. Rhyu, Y. Kim, H. Cho, J.H. Wang, "Clinical advantages of image-free navigation system using surface-based registration in anatomical

anterior cruciate ligament reconstruction", Knee Surgery, Sports Traumatology, Arthroscopy, Vol. 24, No. 11, pp. 3556-3564, Nov. 2016

9. J.G. Seo*, S.M. Kim, J.M. Shin, Y. Kim*, B.H. Lee, "Safety of simultaneous bilateral total knee arthroplasty using an extramedullary referencing system: results from

2098 consecutive patients", Archives of Orthopaedic and Trauma Surgery, Vol. 136, No. 11, pp. 1615-1621, Nov. 2016

10. Y. Kim, Y.H. Na, L. Xing, R. Lee, S. Park, "Automatic deformable surface registration for medical applications by radial basis function-based robust point-matching",

Computers in Biology and Medicine, Vol. 77, No. 1, pp. 173-181, Oct. 2016

11. S.H. Park, S.W. Moon, B.H. Lee, S. Park, Y. Kim, D. Lee, S. Lim, J.H. Wang, "Arthroscopically blind anatomical anterior cruciate ligament reconstruction using only

navigation guidance: a cadaveric study", The Knee, DOI: 10.1016/j.knee.2016.02.020, Jul. 2016 (Epub ahead of print)

12. Y. Kim, W. Kim, D. Lee, "3D Inspection by Registration of CT and Dual X-ray Images", Journal of Int’l Society for Simulation Surgery, Vol. 3, No. 1, pp. 16-21, Jun.

2016

13. J.P. Yoon, S.W. Chung, J. Kim, H.S. Kim, H.J. Lee, W.J. Jeong, K.S. Oh, D.O. Lee, A. Seo, Y. Kim, "Intra-articular injection, subacromial injection, and

hydrodilatation for primary frozen shoulder: a randomized clinical trial", Journal of Shoulder and Elbow Surgery, Vol. 25, No. 3, pp. 376-383, Mar. 2016

14. J. Charton, L. Kim, Y. Kim, "Boolean operations by a robust, exact, and simple method between two colliding shells". Journal of Advanced Mechanical Design,

Systems, and Manufacturing, (Accepted)

15. E. Shim, Y. Kim, D. Lee, B.H. Lee, S. Woo, K. Lee, “2D-3D registration for 3D analysis of lower limb alignment in a weight-bearing condition”, Applied Mathematics –

Journal of Chinese Universities (Accepted)

44

Thank you!

45

Contact Information

Youngjun Kim, Ph.D.

- Center for Bionics, Korea Institute of Science and Technology

- E-mail: junekim@kist.re.kr

- Tel.: 02-958-5606

- C.P.: 010-5234-5378