7851-7900 of 10000 results (73ms)
2017-11-28 §
15:39 <jynus@tin> Synchronized wmf-config/db-eqiad.php: depool db1110 (duration: 00m 44s) [production]
15:38 <bblack> powered off mw1329 [production]
15:06 <marostegui> Compress s5 on db1099 [production]
15:04 <ladsgroup@tin> Synchronized wmf-config/InitialiseSettings.php: Revert "Comply wikidata with new ores thresholds" (duration: 00m 45s) [production]
14:56 <marostegui> Deploy schema change on s3 on dbstore1002 - T174569 [production]
14:45 <gehel@tin> Finished deploy [kartotherian/deploy@cb9b1ef]: testing new kartotherian packaging on maps-test2003 (duration: 00m 02s) [production]
14:45 <gehel@tin> Started deploy [kartotherian/deploy@cb9b1ef]: testing new kartotherian packaging on maps-test2003 [production]
14:41 <otto@tin> Finished deploy [eventlogging/analytics@c464b8c]: Fixing bug where userAgent set by client producer was not used T178440 (duration: 00m 02s) [production]
14:40 <otto@tin> Started deploy [eventlogging/analytics@c464b8c]: Fixing bug where userAgent set by client producer was not used T178440 [production]
14:40 <otto@tin> Started deploy [eventlogging/analytics@c464b8c]: Fixing bug where userAgent set by client producer was not used [production]
14:39 <gehel@tin> Finished deploy [kartotherian/deploy@cb9b1ef]: testing new kartotherian packaging on maps-test2003 (duration: 00m 18s) [production]
14:39 <gehel@tin> Started deploy [kartotherian/deploy@cb9b1ef]: testing new kartotherian packaging on maps-test2003 [production]
14:37 <marostegui> Stop MySQL on db1055 to clone db1099:3311 - T178359 [production]
14:34 <gehel@tin> Finished deploy [kartotherian/deploy@55c5da4]: (no justification provided) (duration: 00m 11s) [production]
14:34 <gehel@tin> Started deploy [kartotherian/deploy@55c5da4]: (no justification provided) [production]
14:21 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1055 - T178359 (duration: 00m 44s) [production]
14:17 <elukey> reboot kafka10[12-22] for kernel + jvm updates - T179943 [production]
14:03 <elukey> reboot kafka200[123] for kernel + jvm updates - T179943 [production]
12:25 <godog> cleanup wanobjectcache metrics with hashes - T178531 [production]
12:14 <godog> bounce carbon-frontend-relay after https://gerrit.wikimedia.org/r/393749 [production]
11:25 <hoo> Manually re-started Wikidata JSON dumps on snapshot1007, got stuck after db1082 went down. [production]
10:55 <jynus> stop db1044 replication for db1095 master switchover [production]
10:49 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1082. Depool db1044 (duration: 00m 44s) [production]
10:44 <mobrovac@tin> Synchronized wmf-config/jobqueue.php: Process wikibase-addUsagesForPage only via EventBus - T175212 (duration: 00m 44s) [production]
10:08 <ppchelko@tin> Finished deploy [cpjobqueue/deploy@c4b9e16]: Enable wikibase-addUsagesForPage with low concurrency (duration: 00m 29s) [production]
10:07 <ppchelko@tin> Started deploy [cpjobqueue/deploy@c4b9e16]: Enable wikibase-addUsagesForPage with low concurrency [production]
09:35 <jynus> restart db1087 for maintenance [production]
09:22 <jynus> restart db1082 [production]
09:19 <godog> unmask and restart restbase1007-b - T179422 [production]
09:15 <moritzm> installibg libxml-libxml-perl security updates on trusty (Debian already fixed) [production]
09:11 <ppchelko@tin> Finished deploy [cpjobqueue/deploy@b0b1793]: Remove double-processing created for consumer group renaming (duration: 00m 28s) [production]
09:10 <ppchelko@tin> Started deploy [cpjobqueue/deploy@b0b1793]: Remove double-processing created for consumer group renaming [production]
09:04 <marostegui> Drop database log from dbstore1002 - T156844 [production]
08:57 <ppchelko@tin> Finished deploy [cpjobqueue/deploy@2212086]: Move enabled jobs config to vars.yaml (duration: 00m 39s) [production]
08:56 <ppchelko@tin> Started deploy [cpjobqueue/deploy@2212086]: Move enabled jobs config to vars.yaml [production]
08:38 <jynus@tin> Synchronized wmf-config/db-eqiad.php: depool db1082 (duration: 00m 44s) [production]
08:02 <mobrovac> bootstrap restbase1007-b - T179422 [production]
06:39 <marostegui> Stop MySQL on db1099 to copy its content to dbstore1001 and reimage it as multi-instance - T178359 [production]
05:23 <mutante> labtestpuppetmaster2001 - manually purging ganglia things since puppet is disabled [production]
05:05 <mutante> labweb1001/labweb1002: manually purging ganglia package/config/service/unit files because puppet is disabled there (T177225) [production]
04:08 <eileen> update process-control to b40eaec592f5274017630c9783f69ed09, disabling 3 jobs to reduce load for Big english starting: dedupe_civicrm_contacts, omnimail_recipient_load, omnimail_recipient_load_backfill [production]
03:58 <demon@tin> Pruned MediaWiki: 1.31.0-wmf.4 (duration: 03m 13s) [production]
03:31 <mutante> labservices1001/1002,labtestservices2001 - remove pdns_gmetric cronjobs causing cron spam after ganglia decom from lab* [production]
02:20 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.8) (duration: 05m 23s) [production]
2017-11-27 §
22:29 <mutante> labtestnet2001 - re-enabled puppet to decom ganglia, no errors [production]
22:22 <mutante> decom'ing Ganglia from all labtest* hosts - packages removed by puppet etc, might cause some false positives in Icinga but it's ok (T177225) [production]
22:21 <otto@tin> Finished deploy [eventlogging/eventbus@e024af3]: deploy revert checked out at 3df06ab (we caught one). T180017 (duration: 00m 10s) [production]
22:21 <otto@tin> Started deploy [eventlogging/eventbus@e024af3]: deploy revert checked out at 3df06ab (we caught one). T180017 [production]
21:59 <awight> clearing ORES threshold caches for ruwiki, frwiki, wikidatawiki. [production]
21:59 <awight@tin> Synchronized wmf-config/InitialiseSettings.php: Reenable ORES on frwiki, ruwiki, and wikidata; T181006 (duration: 00m 45s) [production]