4351-4400 of 10000 results (40ms)
2017-05-09 §
10:35 <akosiaris@tin> Started deploy [librenms/librenms@259e998]: (no justification provided) [production]
10:34 <elukey> reboot kafka1022 for kernel upgrades [production]
10:09 <elukey> reboot kafka1020 for kernel upgrades [production]
09:57 <moritzm> restarting hhvm on mw1190, deadlocked in HPHP::Treadmill::getAgeOldestRequest [production]
09:41 <marostegui> Stop replication at the same position on db1097 and db2019 - https://phabricator.wikimedia.org/T147166 https://phabricator.wikimedia.org/T130067 [production]
09:37 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1097 - T147166 T130067 (duration: 00m 41s) [production]
09:21 <marostegui> Disable replication codfw > eqiad on s4 - https://phabricator.wikimedia.org/T147166 https://phabricator.wikimedia.org/T130067 [production]
09:12 <marostegui> Run pt-table-checksum on s7.eswiki - T163190 [production]
09:07 <hoo> Removed 2fa from global account Jcornelius (T164682) [production]
08:05 <godog> roll-restart swift proxy for ratelimit middleware - T162793 [production]
07:53 <moritzm> uploaded kubernetes 1.4.2-6 for stretch-wikimedia to apt.wikimedia.org [production]
07:34 <moritzm> removing unneeded rpcbind/nfs-common packages (T106477) [production]
07:31 <marostegui> Stop replication at the same position on db1050 and db2028 - T147166 T130067 [production]
07:27 <marostegui> Disable replication codfw > eqiad on s6 - T147166 T130067 [production]
07:12 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Repool es2019 - T149526 (duration: 00m 39s) [production]
07:11 <elukey> reboot kafka1014 for kernel upgrades [production]
07:01 <_joe_> installing the new version of python-service-checker across the fleet [production]
06:37 <marostegui> Run pt-table-checksum on s7.cawiki - T163190 [production]
06:01 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Repool db2038 - T162539 T163548 (duration: 00m 41s) [production]
05:54 <marostegui> Deploy alter table on wikidatawiki.wb_terms on codfw master db2023 - https://phabricator.wikimedia.org/T162539 - https://phabricator.wikimedia.org/T163548 [production]
02:27 <l10nupdate@tin> ResourceLoader cache refresh completed at Tue May 9 02:27:46 UTC 2017 (duration 5m 58s) [production]
02:21 <l10nupdate@tin> scap sync-l10n completed (1.29.0-wmf.21) (duration: 08m 17s) [production]
2017-05-08 §
23:21 <bd808@tin> Synchronized wmf-config/wikitech.php: Revert "Disable creation of new forms on wikitech" (T53642) (duration: 01m 10s) [production]
22:54 <bd808@tin> Finished deploy [striker/deploy@00e8545]: openstack: Role modifications require global admin rights (T164787) (duration: 00m 27s) [production]
22:54 <bd808@tin> Started deploy [striker/deploy@00e8545]: openstack: Role modifications require global admin rights (T164787) [production]
22:17 <bd808> Deleted 2fa for user Mdann52 on wikitech after verifying account ownership via ssh file creation. T164804 [production]
22:01 <andrewbogott> rebooting labservices1002 to mess with the bios [production]
21:55 <bblack> depooled cp3035 (memory issues - already schedule for FE restart to fix, which will repool when it's reached in the list...) [production]
21:25 <mobrovac@tin> Finished deploy [graphoid/deploy@a288409]: Switched to npm-stored graph-shared, fix mapsnapshot - T164046 (duration: 01m 39s) [production]
21:24 <mobrovac@tin> Started deploy [graphoid/deploy@a288409]: Switched to npm-stored graph-shared, fix mapsnapshot - T164046 [production]
21:22 <mobrovac@tin> Finished deploy [graphoid/deploy@a288409]: Switched to npm-stored graph-shared, fix mapsnapshot - T164046 (duration: 03m 51s) [production]
21:18 <mobrovac@tin> Started deploy [graphoid/deploy@a288409]: Switched to npm-stored graph-shared, fix mapsnapshot - T164046 [production]
21:17 <mobrovac@tin> Finished deploy [graphoid/deploy@a288409]: Switched to npm-stored graph-shared, fix mapsnapshot - T164046 (duration: 00m 38s) [production]
21:17 <mobrovac@tin> Started deploy [graphoid/deploy@a288409]: Switched to npm-stored graph-shared, fix mapsnapshot - T164046 [production]
21:12 <mobrovac@tin> Finished deploy [restbase/deploy@c70a1e1]: Remove the mobile-text end point - T158128 (duration: 06m 23s) [production]
21:06 <mobrovac@tin> Started deploy [restbase/deploy@c70a1e1]: Remove the mobile-text end point - T158128 [production]
21:05 <arlolra@tin> Finished deploy [parsoid/deploy@0459ae3]: Updating Parsoid to 9d8badc8 (duration: 02m 43s) [production]
21:02 <arlolra@tin> Started deploy [parsoid/deploy@0459ae3]: Updating Parsoid to 9d8badc8 [production]
20:48 <arlolra@tin> Finished deploy [parsoid/deploy@0459ae3]: Updating Parsoid to 9d8badc8 (duration: 01m 36s) [production]
20:47 <arlolra@tin> Started deploy [parsoid/deploy@0459ae3]: Updating Parsoid to 9d8badc8 [production]
20:37 <gehel> silencing elasticsearch shard incinga check, recovery after upgrade is going to take a long time - T161908 [production]
20:34 <arlolra@tin> Finished deploy [parsoid/deploy@0459ae3]: Updating Parsoid to 9d8badc8 (duration: 04m 50s) [production]
20:30 <arlolra@tin> Started deploy [parsoid/deploy@0459ae3]: Updating Parsoid to 9d8badc8 [production]
20:27 <gehel> restarted kibana on logstash cluster - T161908 [production]
20:21 <gehel> upgrading kibana on logstash cluster - T161908 [production]
20:02 <gehel> restarting elasticsearch on logstash cluster after upgrade - T161908 [production]
19:47 <gehel> logstash / elasticsearch downtime coming up - T161908 [production]
19:34 <bd808> Deployment of Striker for T162508 complete; will continue debug keystone issue that is preventing Tool Labs membership requests from being approved [production]
19:34 <bblack> restarted varnishxcache service on cp3031, was malfunctioning and sending crazy stats to grafana... [production]
19:28 <gehel> starting ELK (logstash) upgrade - T161908 [production]