1251-1300 of 10000 results (19ms)
2014-03-13 §
08:49 <paravoid> API slowness (affecting Parsoid/VE, among others) there since ~12:00 UTC found and fixed [production]
07:42 <RoanKattouw> Reverted API pybal weights back to original values; apparently makes sense given amount of memory [production]
07:05 <RoanKattouw> Changed pybal weights for eqiad API cluster to # of CPUs on each machine; weights were backwards (machines with fewer CPUs had higher weights) [production]
03:23 <LocalisationUpdate> ResourceLoader cache refresh completed at Thu Mar 13 03:23:13 UTC 2014 (duration 23m 12s) [production]
02:34 <LocalisationUpdate> completed (1.23wmf17) at 2014-03-13 02:33:54+00:00 [production]
02:17 <LocalisationUpdate> completed (1.23wmf16) at 2014-03-13 02:17:46+00:00 [production]
00:14 <awight> updated crm from 445f1f0ce489e65bf3765a23403f5b52b77f23cf to 2353dbcda410645e02dd9f049ff9e1eb341d2f16 [production]
2014-03-12 §
23:50 <mwalker> synchronized php-1.23wmf17/extensions/CentralNotice/ [production]
23:49 <mwalker> synchronized php-1.23wmf16/extensions/CentralNotice/ [production]
23:21 <mwalker> synchronized php-1.23wmf17/extensions/CentralNotice [production]
23:20 <mwalker> synchronized php-1.23wmf16/extensions/CentralNotice [production]
21:57 <bd808> Finished scap: another no-diff scap to test script changes (duration: 03m 25s) [production]
21:54 <bd808> Started scap: another no-diff scap to test script changes [production]
21:54 <ottomata> initiated controlled shutdown of an21, promoting an22 to leader of all partitions [production]
21:51 <bd808> Finished scap: no-diff scap to test script changes (duration: 21m 42s) [production]
21:30 <bd808> Started scap: no-diff scap to test script changes [production]
21:14 <mutante> welcome to root, Chase, added key to root-auth-keys [production]
20:05 <Coren> restarting parsoids as requested by gwicke [production]
20:02 <gwicke> deployed Parsoid 004c7acc with deploy f97820a2; restart todo [production]
19:12 <yurik> synchronized php-1.23wmf16/extensions/ZeroRatedMobileAccess/ [production]
19:11 <yurik> synchronized php-1.23wmf17/extensions/ZeroRatedMobileAccess/ [production]
19:06 <mutante> graceful apache on gallium [production]
17:36 <yurik> synchronized php-1.23wmf17/extensions/ZeroRatedMobileAccess/ [production]
17:27 <yurik> synchronized php-1.23wmf16/extensions/ZeroRatedMobileAccess/ [production]
17:17 <awight> updated crm from f0837090e918b7016c07cb366b33c8d8f0d1c661 to 445f1f0ce489e65bf3765a23403f5b52b77f23cf [production]
17:02 <reedy> synchronized wmf-config/ 'noop' [production]
16:56 <reedy> synchronized docroot and w [production]
16:56 <reedy> synchronized wmf-config/ 'I6f1f3f8af5b97aa0e537fbae308ce27b28071894' [production]
16:10 <ottomata> initiating controlled shutdown of analytics1021 kafka broker to do some load testing and also fix runtime java version [production]
15:56 <cmjohnson1> ms-be1005 going down to fix mgmt [production]
15:06 <hoo> syncing to mw120[1-3] failed [production]
14:58 <hoo> synchronized php-1.23wmf17/extensions/CentralAuth/ 'Fix global account deletion' [production]
14:38 <springle> synchronized wmf-config/db-eqiad.php 's2 db1063 full steam' [production]
14:24 <paravoid> deploying new swift ring @ eqiad, setting weight from 100 to 2000 on all disks [production]
13:01 <cmjohnson1> shutting down and relocating mw1201, mw1202, mw1203 to d5-eqiad [production]
10:41 <maxsem> synchronized wmf-config/mobile.php 'https://gerrit.wikimedia.org/r/118227' [production]
10:17 <springle> started s4 dump for toolserver on db72 /a [production]
09:52 <springle> synchronized wmf-config/db-eqiad.php 's2 pool db1063, warm up' [production]
09:45 <hashar> rerestarting Jenkins [production]
09:16 <hashar> kill -9 of Jenkins since it is unresponsive [production]
09:13 <hashar> restarting Jenkins [production]
09:12 <hashar> Jenkins broken again! Good morning. [production]
06:18 <springle> xtrabackup clone db1018 to db1063 [production]
05:38 <springle> synchronized wmf-config/db-eqiad.php 's1 pool db1061, warm up' [production]
04:52 <springle> synchronized wmf-config/db-eqiad.php 's1 db1051 warm up' [production]
02:43 <LocalisationUpdate> ResourceLoader cache refresh completed at Wed Mar 12 02:42:59 UTC 2014 (duration 42m 58s) [production]
02:32 <springle> synchronized wmf-config/db-eqiad.php 's1 drop load during xtrabackup clone db1051 to db1061' [production]
02:18 <LocalisationUpdate> completed (1.23wmf17) at 2014-03-12 02:18:12+00:00 [production]
02:10 <LocalisationUpdate> completed (1.23wmf16) at 2014-03-12 02:10:25+00:00 [production]
2014-03-11 §
19:36 <mutante> re-deleting salt keys for pmtpa appservers [production]