vít novák: velká data v malém šuplíčku
TRANSCRIPT
První krůčky k udrženívelkých dat v malé krabičce
Víťa NovákHeukreka.cz
Šéfe, nebylo by nějaký železo?
Zdroj: ČT24.cz
- Pořizovací náklady
- Provozní náklady
- Lidské zdroje
- Dost možná by vám to vůbec nepomohlo
Měřte si to
- cachuje se?
- neopakují se dotazy?
- SELECTy max pár ms?
-- (1196.34 ms) SELECT p.`productId`, c.`seoId` FROM CategoryToplist ct JOIN Product p ON ct.`categoryId` = p.`categoryId` JOIN ProductTopFifty ptf p.`productId` = ptf.`productId` JOIN Category c ON p.`categoryId` = c.`categoryId` WHERE p.`displayStatus` = 22 AND ptf.`order` = 1 AND ct.`categoryId` IN (1, 2, ... ) ORDER BY ct.`rank` DESC;
ALTER TABLE `ProductTopFifty` ADD INDEX `order` (`order`);
ALTER TABLE `ProductTopFifty` ADD INDEX `order` (`order`);
-- (925.47 ms) SELECT p.`productId`, c.`seoId` FROM CategoryToplist ct JOIN Product p ON ct.`categoryId` = p.`categoryId` JOIN ProductTopFifty ptf p.`productId` = ptf.`productId` JOIN Category c ON p.`categoryId` = c.`categoryId` WHERE p.`displayStatus` = 22 AND ptf.`order` = 1 AND ct.`categoryId` IN (1, 2, ... ) ORDER BY ct.`rank` DESC;
ALTER TABLE `ProductTopFifty` ADD INDEX `order` (`order`);
-- (925.47 ms) SELECT p.`productId`, c.`seoId` FROM CategoryToplist ct JOIN Product p ON ct.`categoryId` = p.`categoryId` JOIN ProductTopFifty ptf p.`productId` = ptf.`productId` JOIN Category c ON p.`categoryId` = c.`categoryId` WHERE p.`displayStatus` = 22 AND ptf.`order` = 1 AND ct.`categoryId` IN (1, 2, ... ) ORDER BY ct.`rank` DESC;
SELECT STRAIGHT_JOIN p.`productId`, c.`seoId` FROM ProductTopFifty ptf FORCE INDEX(`order`) JOIN Product p ON p.`productId` = ptf.`productId` JOIN CategoryToplist ct ON p.`categoryId` = ct.`categoryId` JOIN Category c ON p.`categoryId` = c.`categoryId` WHERE p.`displayStatus` = 22 AND ptf.`order` = 1 AND ct.`categoryId` IN (1, 2, ... ) ORDER BY ct.`rank` DESC;
ALTER TABLE `ProductTopFifty` ADD INDEX `order` (`order`);
-- (925.47 ms) SELECT p.`productId`, c.`seoId` FROM CategoryToplist ct JOIN Product p ON ct.`categoryId` = p.`categoryId` JOIN ProductTopFifty ptf p.`productId` = ptf.`productId` JOIN Category c ON p.`categoryId` = c.`categoryId` WHERE p.`displayStatus` = 22 AND ptf.`order` = 1 AND ct.`categoryId` IN (1, 2, ... ) ORDER BY ct.`rank` DESC;
-- (7.69 ms) SELECT STRAIGHT_JOIN p.`productId`, c.`seoId` FROM ProductTopFifty ptf FORCE INDEX(`order`) JOIN Product p ON p.`productId` = ptf.`productId` JOIN CategoryToplist ct ON p.`categoryId` = ct.`categoryId` JOIN Category c ON p.`categoryId` = c.`categoryId` WHERE p.`displayStatus` = 22 AND ptf.`order` = 1 AND ct.`categoryId` IN (1, 2, ... ) ORDER BY ct.`rank` DESC;
- dlouhé transakce
- zamykání spousty záznamů
- zamykání ještě více záznamů
Najít a zabít!SELECT (UNIX_TIMESTAMP() - UNIX_TIMESTAMP(trx.trx_started)), trx_weight, trx_query, IF(ps.state = 'Copying to tmp table on disk',1,0), trx_rows_modified, trx_rows_locked FROM INFORMATION_SCHEMA.INNODB_TRX trx LEFT JOIN INFORMATION_SCHEMA.PROCESSLIST ps ON trx.trx_mysql_thread_id = ps.id WHERE (UNIX_TIMESTAMP() - UNIX_TIMESTAMP(trx.trx_started)) >= max_transaction_time AND (trx_state = 'RUNNING');
Najít a zabít!SELECT (UNIX_TIMESTAMP() - UNIX_TIMESTAMP(trx.trx_started)), trx_weight, trx_query, IF(ps.state = 'Copying to tmp table on disk',1,0), trx_rows_modified, trx_rows_locked FROM INFORMATION_SCHEMA.INNODB_TRX trx LEFT JOIN INFORMATION_SCHEMA.PROCESSLIST ps ON trx.trx_mysql_thread_id = ps.id WHERE (UNIX_TIMESTAMP() - UNIX_TIMESTAMP(trx.trx_started)) >= max_transaction_time AND (trx_state = 'RUNNING');
-----------------------------------To: [email protected]: [email protected]: Teda ty mas ale transakci! -----------------------------------array ( 'id' => '1240560', 'dbHost' => 'productionserver1.h.cz', 'ts' => '2013-08-18 00:06:23', 'thd' => '25290427', 'statement' => 'DELETE FROM XmlImportStatus WHERE timeStarted < (NOW() - INTERVAL 48 HOUR) LIMIT 10000', 'runtime' => '33', 'tmp_table_on_disk' => '0', 'rows_locked' => '2809', 'rows_modified' => '67493', )
function deleteSomething(){ DB::query("DELETE FROM XmlImportStatus WHERE timeStarted < (NOW() - INTERVAL 48 HOUR) LIMIT 10000");}
function deleteSomething(){ DB::query("DELETE FROM XmlImportStatus WHERE timeStarted < (NOW() - INTERVAL 48 HOUR) LIMIT 10000");}
function deleteSomething(){ do { $affected = DB::query("DELETE FROM XmlImportStatus WHERE timeStarted < (NOW() - INTERVAL 48 HOUR) LIMIT 1000"); } while (0 < $affected);}
Vytmav mi to! Pro hodnocení přednášky skenuj:
Podpoř dobrou věc: Heureka Boys na http://movember.comDej se mezi nás! http://info.heureka.cz/kariera/
Vytmav mi to! Pro hodnocení přednášky skenuj:
Podpoř dobrou věc: Heureka Boys na http://movember.comDej se mezi nás! http://info.heureka.cz/kariera/