|
2017-11-28
§
|
| 08:02 |
<mobrovac> |
bootstrap restbase1007-b - T179422 |
[production] |
| 06:39 |
<marostegui> |
Stop MySQL on db1099 to copy its content to dbstore1001 and reimage it as multi-instance - T178359 |
[production] |
| 05:23 |
<mutante> |
labtestpuppetmaster2001 - manually purging ganglia things since puppet is disabled |
[production] |
| 05:05 |
<mutante> |
labweb1001/labweb1002: manually purging ganglia package/config/service/unit files because puppet is disabled there (T177225) |
[production] |
| 04:08 |
<eileen> |
update process-control to b40eaec592f5274017630c9783f69ed09, disabling 3 jobs to reduce load for Big english starting: dedupe_civicrm_contacts, omnimail_recipient_load, omnimail_recipient_load_backfill |
[production] |
| 03:58 |
<demon@tin> |
Pruned MediaWiki: 1.31.0-wmf.4 (duration: 03m 13s) |
[production] |
| 03:31 |
<mutante> |
labservices1001/1002,labtestservices2001 - remove pdns_gmetric cronjobs causing cron spam after ganglia decom from lab* |
[production] |
| 02:20 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.8) (duration: 05m 23s) |
[production] |
|
2017-11-27
§
|
| 22:29 |
<mutante> |
labtestnet2001 - re-enabled puppet to decom ganglia, no errors |
[production] |
| 22:22 |
<mutante> |
decom'ing Ganglia from all labtest* hosts - packages removed by puppet etc, might cause some false positives in Icinga but it's ok (T177225) |
[production] |
| 22:21 |
<otto@tin> |
Finished deploy [eventlogging/eventbus@e024af3]: deploy revert checked out at 3df06ab (we caught one). T180017 (duration: 00m 10s) |
[production] |
| 22:21 |
<otto@tin> |
Started deploy [eventlogging/eventbus@e024af3]: deploy revert checked out at 3df06ab (we caught one). T180017 |
[production] |
| 21:59 |
<awight> |
clearing ORES threshold caches for ruwiki, frwiki, wikidatawiki. |
[production] |
| 21:59 |
<awight@tin> |
Synchronized wmf-config/InitialiseSettings.php: Reenable ORES on frwiki, ruwiki, and wikidata; T181006 (duration: 00m 45s) |
[production] |
| 21:53 |
<awight@tin> |
Finished deploy [ores/deploy@e58bfbf]: (codfw) Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 (duration: 05m 06s) |
[production] |
| 21:47 |
<awight@tin> |
Started deploy [ores/deploy@e58bfbf]: (codfw) Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 |
[production] |
| 21:46 |
<awight@tin> |
Finished deploy [ores/deploy@e58bfbf]: Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 (duration: 12m 55s) |
[production] |
| 21:33 |
<awight@tin> |
Started deploy [ores/deploy@e58bfbf]: Update ORES for impossible threshold handling and new wikidata editquality model, T179711 T180686 T180450 |
[production] |
| 21:23 |
<awight@tin> |
Synchronized php-1.31.0-wmf.8/extensions/ORES: ORES error handling for bad thresholds, T181191 (duration: 00m 46s) |
[production] |
| 21:11 |
<awight@tin> |
Synchronized wmf-config/InitialiseSettings.php: Temporarily disable ORES on wikidata (duration: 00m 45s) |
[production] |
| 21:10 |
<awight> |
Previous “rollback ORES” was from a stale screen session, no actions were taken today. |
[production] |
| 21:08 |
<awight@tin> |
Finished deploy [ores/deploy@82a13ae]: Rollback ORES (take 3); 181006 (duration: 9942m 42s) |
[production] |
| 20:21 |
<demon@tin> |
i lied, that was for wgStyleVersion removal |
[production] |
| 20:20 |
<demon@tin> |
Synchronized wmf-config/CommonSettings.php: cli sapi fixes (duration: 00m 45s) |
[production] |
| 20:17 |
<ejegg> |
updated payments-wiki from fea6b375affc629f3dbce71aa348704b307e7a94 to f594dfa763292b2f8785b6aa83d6b50ef8cae64e |
[production] |
| 20:03 |
<kaldari@tin> |
Synchronized wmf-config/InitialiseSettings-labs.php: (no justification provided) (duration: 00m 45s) |
[production] |
| 20:02 |
<kaldari@tin> |
Synchronized wmf-config/CommonSettings.php: (no justification provided) (duration: 00m 45s) |
[production] |
| 20:01 |
<kaldari@tin> |
Synchronized wmf-config/InitialiseSettings.php: (no justification provided) (duration: 00m 46s) |
[production] |
| 19:54 |
<bblack> |
crN-ulsfo: remove lvs400[1-4] from PyBal BGP neighbors list |
[production] |
| 19:42 |
<catrope@tin> |
Synchronized php-1.31.0-wmf.8/resources/src/mediawiki.rcfilters/mw.rcfilters.UriProcessor.js: T181100 (duration: 00m 45s) |
[production] |
| 19:41 |
<demon@tin> |
Synchronized wmf-config/InitialiseSettings.php: babel officewiki thing (duration: 00m 45s) |
[production] |
| 19:39 |
<catrope@tin> |
Synchronized php-1.31.0-wmf.8/includes/specials/SpecialRecentchangeslinked.php: T181100 (duration: 00m 45s) |
[production] |
| 19:34 |
<otto@tin> |
Finished deploy [eventlogging/eventbus@3df06ab]: Temp deploy to kafka1001 only to catch bug: T180017 (duration: 00m 12s) |
[production] |
| 19:34 |
<otto@tin> |
Started deploy [eventlogging/eventbus@3df06ab]: Temp deploy to kafka1001 only to catch bug: T180017 |
[production] |
| 19:21 |
<catrope@tin> |
Synchronized wmf-config/Wikibase.php: T176903 (duration: 00m 45s) |
[production] |
| 19:17 |
<AaronSchulz> |
Removed more bogus md5 wanobjecache/ metric from graphite[12]001 |
[production] |
| 19:11 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: T179241 (duration: 00m 45s) |
[production] |
| 19:02 |
<volans> |
restarted ircecho |
[production] |
| 18:51 |
<ejegg> |
updated payments-wiki from 6b3019b1f18b4d6cd1705a49e95005124435e3d2 to fea6b375affc629f3dbce71aa348704b307e7a94 |
[production] |
| 18:50 |
<dcausse> |
elastic/cirrus: reindexing english group2 wikis: T179945 |
[production] |
| 18:46 |
<Zoranzoki21> |
test |
[production] |
| 18:45 |
<volans> |
stopped ircecho temporary to avoid spam |
[production] |
| 18:14 |
<XioNoX> |
pushing v6 gre firewall filter |
[production] |
| 18:12 |
<gehel@tin> |
Finished deploy [wdqs/wdqs@7ad20f3]: (no justification provided) (duration: 02m 10s) |
[production] |
| 18:11 |
<gehel> |
deploying blazegraph + GUI updates |
[production] |
| 18:10 |
<gehel@tin> |
Started deploy [wdqs/wdqs@7ad20f3]: (no justification provided) |
[production] |
| 18:08 |
<hoo> |
Killed truthy nt dumpers stuck at 100% CPU (from last week) on snapshot1007 (T181385) |
[production] |
| 18:06 |
<akosiaris> |
poweroff ganeti1005, ganeti1006 T181121 |
[production] |
| 18:05 |
<bd808> |
Sending Toolforge survey 1 week reminder emails from silver for T177126 |
[production] |
| 16:07 |
<bblack> |
lvs400x - puppet disabled for https://gerrit.wikimedia.org/r/#/c/393610 |
[production] |