3351-3400 of 10000 results (50ms)
2017-07-10 ยง
15:00 <moritzm> installing apache security updates on app server canaries [production]
14:40 <andrewbogott> rebooting labvirt1015-1018 for kernel updates [production]
14:23 <marostegui> Deploy alter table on db1059 - T168661 [production]
14:22 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1059 - T168661 (duration: 00m 41s) [production]
14:20 <marostegui@tin> scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) [production]
14:14 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1091 - T168661 (duration: 00m 42s) [production]
14:13 <moritzm> reimaging mw2118 (video scaler) to jessie [production]
14:12 <TabbyCat> mwscript updateCollation.php --wiki=frwiktionary --previous-collation=uppercase is being running by zfilipin to finish T169810 [production]
14:11 <zeljkof> EU SWAT finished [production]
14:01 <dcausse> elastic@eqiad unbanning elastic1018 & elastic1021 [production]
13:49 <chasemp> labstore2003:~# umount -fl /srv/backup/tools (for T169774 recovery) [production]
13:44 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:363740|Set $wgCategoryCollation to uca-default for fr.wiktionary (T169810)]] (duration: 00m 42s) [production]
13:43 <dcausse> elastic@eqiad banning elastic1018 & elastic1021 to rebalance heavy shards [production]
13:32 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:364131|Add import sources for specieswiki (T170094)]] (duration: 00m 43s) [production]
13:29 <moritzm> installing graphite2 security updates (image lib) [production]
13:28 <marostegui> Disable puppet on db1102 to run check_private_data - T153743 [production]
13:20 <zfilipin@tin> Synchronized wmf-config/CirrusSearch-common.php: SWAT: [[gerrit:364183|[cirrus] Enable the token_count_router only for chinese (T169498)]] (duration: 00m 43s) [production]
13:06 <milimetric@tin> Finished deploy [analytics/refinery@c22eb93]: Update Sqoop with better parallelism (duration: 02m 54s) [production]
13:04 <milimetric@tin> Started deploy [analytics/refinery@c22eb93]: Update Sqoop with better parallelism [production]
12:58 <marostegui> Run redact_sanitarium on s2 and s6 - db1102 - T153743 [production]
12:26 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: db1079 to become master for sanitarium3 - T153743 (duration: 00m 41s) [production]
12:23 <marostegui@tin> scap failed: average error rate on 2/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) [production]
12:22 <moritzm> installing xorg-server security updates [production]
12:22 <marostegui@tin> scap failed: average error rate on 4/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) [production]
12:12 <marostegui> Deploy alter table on db1091 - T168661 [production]
12:11 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1091 - T168661 (duration: 00m 42s) [production]
12:02 <marostegui> Upgrade db1102 to 10.1 and enable rbr triggers - T153743 [production]
12:02 <moritzm> installing bind security updates (we only have client libs/tools installed) [production]
11:39 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Restore original weight for db1097 - T168661 (duration: 00m 42s) [production]
11:36 <marostegui> Stop MySQL on db1102 for maintenance - T153743 [production]
11:16 <moritzm> installing libgcrypt and expat security updates [production]
11:16 <kartik@tin> Finished deploy [cxserver/deploy@c209bec]: Update cxserver to 3375da5 (duration: 02m 49s) [production]
11:13 <kartik@tin> Started deploy [cxserver/deploy@c209bec]: Update cxserver to 3375da5 [production]
10:28 <addshore> WMDE Summer campaign deploy slot DONE [production]
10:28 <addshore@tin> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:364184|WMDE Summer campaign - Add logging]] (fix spacing) NOOP (duration: 00m 43s) [production]
10:22 <addshore@tin> Synchronized php-1.30.0-wmf.7/extensions/WikimediaEvents: [[gerrit:364172|WMDE Summer campaign - Add hook]] (duration: 00m 42s) [production]
10:21 <addshore@tin> Synchronized php-1.30.0-wmf.7/extensions/CentralAuth: CentralAuth (undeployed patches) [[gerrit:363892]], [[gerrit:363893]], [[gerrit:363891]] & revert [[gerrit:364182]] T169261 (duration: 00m 47s) [production]
10:13 <addshore> reverting https://gerrit.wikimedia.org/r/#/c/363891 as it is sitting on tin undeployed T169261 [production]
09:59 <moritzm> rebooting mc2* servers for kernel update [production]
09:54 <addshore@tin> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:362380|WMDE Summer campaign - Add logging]] (duration: 00m 45s) [production]
09:10 <marostegui> Compress innodb on wikidata on dbstore2001 [production]
09:00 <moritzm> rebooting mw1168 (video scaler) for kernel update [production]
08:52 <moritzm> rebooting mwlog2001 for kernel update [production]
08:35 <moritzm> rebooting ms1001 for kernel update [production]
08:29 <moritzm> rebooting francium for kernel update [production]
08:17 <marostegui> Deploy alter table on db1097 - T168661 [production]
08:17 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1097 - T168661 (duration: 00m 46s) [production]
08:16 <marostegui@tin> scap failed: average error rate on 2/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) [production]
08:03 <marostegui> Drop database l10nwiki on s2 - T119811 [production]
07:53 <moritzm> rebooting hafnium for kernel update [production]