計算論的学習理論チーム 畑埜 晃平Computational Learning Theory Team

Kohei Hatano

Organization

Kohei Hatano (Team Leader, Kyushu) Ken-ichiro Moridomi (Kyushu, ph.D student)

Collaborators Daiki Suehiro (Kyushu) Eiji Takimoto (Kyushu)

Summary

Motivation:

Current Deep Learning is good at dealing with continuous data,

but not good for data & hypotheses with discrete/ structural constraints E.g., compounds in material science, DNA sequeces,

Goal: Develop algorithms for optimization/prediction problems

with discrete/struction constraintsBottleneck︓Candidates are exponentially many （naïve method fails）Situation: Ad-hoc / heuristics only

Current Projects

Theory & Algorithms for local patterns

Online Prediction with LogDet Regularizer

Learning over compressed data (ongoing)

Robust linear programming boosting (ongoing)

Decision diagrams for job scheduling with precedence constraints (ongoing)

Our approach: Organized design based on discrete/offline/online/bandit optimization

Example of our results: Machine Learning(boosting) on compressed data[WALCOM 18]

Key idea: Data compression +online optimization under constraintsSpace efficiency: Can learn while compressing data e.g. a9a data: 70%

reductionContribution: New compression-based ML

Future work: Develop black-box optimization methods under discrete constraints

Theory & Algorithm for Local Patterns

Online Prediction with LogDet Regularizer

AdaBoost*Input: , (pos. & neg. data)

Our method

Compressed data:

hypothesis:

Computation time does not depend on | |, | |, but on | |

Publications1. Ken-ichiro Moridomi, Kohei Hatano, and Eiji Takimoto, “Online linear optimization with the log-determinant

regularizer,” IEICE Transactions on Information and Systems, accepted, 2018.2. Takahiro Fujita, Kohei Hatano, and Eiji Takimoto, “Online Combinatorial Optimization with Multiple Projections and

Its Application to Scheduling Problem,” IEICE Transactions on Information and Systems, accepted, 2018.3. Daiki Suehiro, Kohei Hatano, and Eiji Takimoto, “Efficient Reformulation of 1-norm Ranking SVM,” IEICE

Transactions on Information and Systems, accpted，Vol.E101-D,No.3, 2018.4. Takahiro Fujita, Kohei Hatano and Eiji Takimoto, “Boosting over non-deterministic ZDDs,” WALCOM2018,

accepted, 2018.5. Ken-ichiro Moridomi, Kohei Hatano, and Eiji Takimoto, “Tighter generalization bounds for matrix completion via

factorization into constrained matrices,” IEICE Transactions on Information and Systems, conditionally accepted, 2018.

6. Daiki Suehiro, Kohei Hatano, Eiji Takimoto, Shuji Yamamoto, Kenichi Bannai and Akiko Takeda, “Learning theory and algorithms for shapelets and other local features”, NIPS2017 Time-Series Workshop, accepted, 2017.

Boosting over ZDDs

Online Prediction with LogDet Regularizer (2)

Theory & Algorithm for Local Patterns (2)

Online Prediction with LogDet Regularizer (3)

Naturally generalizes shapelets in time-seriesApplicable to other domains with local patterns

Theory: tight generalization guaranteeFormulation: new sparse (non-convex)

optimizationjustifies shapelet-based methods as wellcan be solved in a principled way

(Boosting-like method based on DC programming)

Contribution:New compression-base ML techniquebased on combinatorial online prediction

Collaboration with Bannai & Takeda Teams