2001-2050 of 10000 results (49ms)
2017-11-28 §
06:39 <marostegui> Stop MySQL on db1099 to copy its content to dbstore1001 and reimage it as multi-instance - T178359 [production]
05:23 <mutante> labtestpuppetmaster2001 - manually purging ganglia things since puppet is disabled [production]
05:05 <mutante> labweb1001/labweb1002: manually purging ganglia package/config/service/unit files because puppet is disabled there (T177225) [production]
04:08 <eileen> update process-control to b40eaec592f5274017630c9783f69ed09, disabling 3 jobs to reduce load for Big english starting: dedupe_civicrm_contacts, omnimail_recipient_load, omnimail_recipient_load_backfill [production]
03:58 <demon@tin> Pruned MediaWiki: 1.31.0-wmf.4 (duration: 03m 13s) [production]
03:31 <mutante> labservices1001/1002,labtestservices2001 - remove pdns_gmetric cronjobs causing cron spam after ganglia decom from lab* [production]
02:20 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.8) (duration: 05m 23s) [production]
2017-11-27 §
22:29 <mutante> labtestnet2001 - re-enabled puppet to decom ganglia, no errors [production]
22:22 <mutante> decom'ing Ganglia from all labtest* hosts - packages removed by puppet etc, might cause some false positives in Icinga but it's ok (T177225) [production]
22:21 <otto@tin> Finished deploy [eventlogging/eventbus@e024af3]: deploy revert checked out at 3df06ab (we caught one). T180017 (duration: 00m 10s) [production]
22:21 <otto@tin> Started deploy [eventlogging/eventbus@e024af3]: deploy revert checked out at 3df06ab (we caught one). T180017 [production]
21:59 <awight> clearing ORES threshold caches for ruwiki, frwiki, wikidatawiki. [production]
21:59 <awight@tin> Synchronized wmf-config/InitialiseSettings.php: Reenable ORES on frwiki, ruwiki, and wikidata; T181006 (duration: 00m 45s) [production]
21:53 <awight@tin> Finished deploy [ores/deploy@e58bfbf]: (codfw) Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 (duration: 05m 06s) [production]
21:47 <awight@tin> Started deploy [ores/deploy@e58bfbf]: (codfw) Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 [production]
21:46 <awight@tin> Finished deploy [ores/deploy@e58bfbf]: Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 (duration: 12m 55s) [production]
21:33 <awight@tin> Started deploy [ores/deploy@e58bfbf]: Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 [production]
21:23 <awight@tin> Synchronized php-1.31.0-wmf.8/extensions/ORES: ORES error handling for bad thresholds, T181191 (duration: 00m 46s) [production]
21:11 <awight@tin> Synchronized wmf-config/InitialiseSettings.php: Temporarily disable ORES on wikidata (duration: 00m 45s) [production]
21:10 <awight> Previous “rollback ORES” was from a stale screen session, no actions were taken today. [production]
21:08 <awight@tin> Finished deploy [ores/deploy@82a13ae]: Rollback ORES (take 3); 181006 (duration: 9942m 42s) [production]
20:21 <demon@tin> i lied, that was for wgStyleVersion removal [production]
20:20 <demon@tin> Synchronized wmf-config/CommonSettings.php: cli sapi fixes (duration: 00m 45s) [production]
20:17 <ejegg> updated payments-wiki from fea6b375affc629f3dbce71aa348704b307e7a94 to f594dfa763292b2f8785b6aa83d6b50ef8cae64e [production]
20:03 <kaldari@tin> Synchronized wmf-config/InitialiseSettings-labs.php: (no justification provided) (duration: 00m 45s) [production]
20:02 <kaldari@tin> Synchronized wmf-config/CommonSettings.php: (no justification provided) (duration: 00m 45s) [production]
20:01 <kaldari@tin> Synchronized wmf-config/InitialiseSettings.php: (no justification provided) (duration: 00m 46s) [production]
19:54 <bblack> crN-ulsfo: remove lvs400[1-4] from PyBal BGP neighbors list [production]
19:42 <catrope@tin> Synchronized php-1.31.0-wmf.8/resources/src/mediawiki.rcfilters/mw.rcfilters.UriProcessor.js: T181100 (duration: 00m 45s) [production]
19:41 <demon@tin> Synchronized wmf-config/InitialiseSettings.php: babel officewiki thing (duration: 00m 45s) [production]
19:39 <catrope@tin> Synchronized php-1.31.0-wmf.8/includes/specials/SpecialRecentchangeslinked.php: T181100 (duration: 00m 45s) [production]
19:34 <otto@tin> Finished deploy [eventlogging/eventbus@3df06ab]: Temp deploy to kafka1001 only to catch bug: T180017 (duration: 00m 12s) [production]
19:34 <otto@tin> Started deploy [eventlogging/eventbus@3df06ab]: Temp deploy to kafka1001 only to catch bug: T180017 [production]
19:21 <catrope@tin> Synchronized wmf-config/Wikibase.php: T176903 (duration: 00m 45s) [production]
19:17 <AaronSchulz> Removed more bogus md5 wanobjecache/ metric from graphite[12]001 [production]
19:11 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: T179241 (duration: 00m 45s) [production]
19:02 <volans> restarted ircecho [production]
18:51 <ejegg> updated payments-wiki from 6b3019b1f18b4d6cd1705a49e95005124435e3d2 to fea6b375affc629f3dbce71aa348704b307e7a94 [production]
18:50 <dcausse> elastic/cirrus: reindexing english group2 wikis: T179945 [production]
18:46 <Zoranzoki21> test [production]
18:45 <volans> stopped ircecho temporary to avoid spam [production]
18:14 <XioNoX> pushing v6 gre firewall filter [production]
18:12 <gehel@tin> Finished deploy [wdqs/wdqs@7ad20f3]: (no justification provided) (duration: 02m 10s) [production]
18:11 <gehel> deploying blazegraph + GUI updates [production]
18:10 <gehel@tin> Started deploy [wdqs/wdqs@7ad20f3]: (no justification provided) [production]
18:08 <hoo> Killed truthy nt dumpers stuck at 100% CPU (from last week) on snapshot1007 (T181385) [production]
18:06 <akosiaris> poweroff ganeti1005, ganeti1006 T181121 [production]
18:05 <bd808> Sending Toolforge survey 1 week reminder emails from silver for T177126 [production]
16:07 <bblack> lvs400x - puppet disabled for https://gerrit.wikimedia.org/r/#/c/393610 [production]
15:52 <herron> beginning canary cutover/deployment of codfw prometheus servers to codfw puppet 4 puppetmasters [production]