5601-5650 of 10000 results (62ms)
2018-06-24 §
22:26 <gehel> restarting wdqs1004 - T198042 [production]
21:51 <gehel> restarting wdqs1005 after taking a few thread dumps - T198042 [production]
19:41 <gehel> restart blazegraph on wdqs1004 [production]
16:09 <godog> restart wdqs-updater on wdqs1005 [production]
15:40 <godog> restart wdqs-blazegraph on wdqs1005 [production]
15:40 <jynus> restart graphite2001 [production]
15:35 <godog> systemctl restart wdqs-blazegraph on wdqs1004 [production]
15:33 <godog> restart-wdqs on wdqs1004 [production]
2018-06-22 §
19:54 <jynus> applying transaction manually on dbstore1002 due to weird bug with savepoint on tokudb+image table [production]
2018-06-21 §
21:38 <ebernhardson> banned elastic1036 from search cluster, waited for all load to shift away, and unbanned [production]
16:38 <ejegg> updated CiviCRM from 349d43eb8b to e2c8ddd70e [production]
13:28 <vgutierrez> Bump AES128-SHA pageview replacement to 4% [production]
12:49 <chasemp> remove labvirt1019 canary to start debug of network [production]
11:08 <hashar> Refreshed operations-puppet-tests-docker jenkins job to a new Docker container build that includes isc-dhcp-server | https://gerrit.wikimedia.org/r/c/integration/config/+/441367 [production]
02:51 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.999) (duration: 07m 40s) [production]
02:33 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.8) (duration: 14m 12s) [production]
2018-06-20 §
21:18 <ejegg> updated 'domain' settings in CiviCRM so name is now 'Wikimedia Foundation' and description is 'donate.wikimedia.org' [production]
20:08 <bearND> rolled back "Update mobileapps to 9d856ec" (was just on canary) [production]
20:07 <bsitzmann@deploy1001> Finished deploy [mobileapps/deploy@3420e67]: Update mobileapps to 9d856ec (duration: 03m 51s) [production]
20:03 <bsitzmann@deploy1001> Started deploy [mobileapps/deploy@3420e67]: Update mobileapps to 9d856ec [production]
19:54 <ottomata> removed Kafka MirrorMaker from kafka10(12|13|14) [production]
16:01 <sbisson@deploy1001> Finished deploy [kartotherian/deploy@9af9191]: Kartotherian: make Pl fallback to EN (duration: 03m 08s) [production]
15:58 <sbisson@deploy1001> Started deploy [kartotherian/deploy@9af9191]: Kartotherian: make Pl fallback to EN [production]
15:57 <sbisson@deploy1001> Finished deploy [kartotherian/deploy@9af9191]: Kartotherian: make Pl fallback to EN (duration: 00m 27s) [production]
15:56 <sbisson@deploy1001> Started deploy [kartotherian/deploy@9af9191]: Kartotherian: make Pl fallback to EN [production]
15:52 <sbisson@deploy1001> Finished deploy [kartotherian/deploy@9af9191]: (no justification provided) (duration: 03m 36s) [production]
15:49 <sbisson@deploy1001> Started deploy [kartotherian/deploy@9af9191]: (no justification provided) [production]
14:59 <dcausse> disk space issue on elastic1020 is due to shard rebalancing (currently receiving 2 enwiki_general shards but removing one wikidatawiki_content) [production]
14:07 <imarlier@deploy1001> Finished deploy [performance/navtiming@742edb0]: (no justification provided) (duration: 00m 04s) [production]
14:07 <imarlier@deploy1001> Started deploy [performance/navtiming@742edb0]: (no justification provided) [production]
14:03 <imarlier@deploy1001> Finished deploy [performance/navtiming@8914e26]: (no justification provided) (duration: 00m 05s) [production]
14:03 <imarlier@deploy1001> Started deploy [performance/navtiming@8914e26]: (no justification provided) [production]
13:43 <imarlier@deploy1001> Finished deploy [performance/navtiming@995cb0f]: (no justification provided) (duration: 00m 05s) [production]
13:43 <imarlier@deploy1001> Started deploy [performance/navtiming@995cb0f]: (no justification provided) [production]
13:38 <anomie> Re-running populateExternallinksIndex60.php on plwiki and ptwiki for [[phab:T59176]] (initial run collided with the s2 master switch). [production]
08:57 <_joe_> removing user.log as well on tegmen [production]
08:54 <_joe_> running logrotate on tegmen [production]
08:54 <_joe_> killall nsca on tegmen [production]
08:51 <_joe_> restarting nsca daemon on tegmen (gone wild, hundreds of subprocesses [production]
08:47 <_joe_> removing user.log.1 and messages.log.1 on tegmen to save some space [production]
07:40 <_joe_> powercycled mw1272, down since yesterday [production]
07:29 <hashar> deploy1001: rebased php-1.32.0-wmf.8/extensions/Translate to catch up with a non production merged change ( https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/441127 ). [production]
02:20 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.8) (duration: 08m 17s) [production]
2018-06-19 §
16:10 <mobrovac@deploy1001> Finished deploy [proton/deploy@43af7d9]: (no justification provided) (duration: 00m 27s) [production]
16:09 <mobrovac@deploy1001> Started deploy [proton/deploy@43af7d9]: (no justification provided) [production]
14:13 <herron> re-enabling puppet agents [production]
14:11 <herron> increased client_max_body_size on puppetdb nginx frontends from 30m to 60m https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/441044/ [production]
14:06 <herron> temporarily disabling puppet agents for deploy of https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/441044/ [production]
12:41 <_joe_> hard reboot of ms-be1019 - unable to ssh, console showing i/o errors only [production]
04:56 <ebernhardson> unban elastic1035 [production]