|
2018-10-23
§
|
| 11:23 |
<zeljkof> |
EU SWAT finished |
[production] |
| 11:20 |
<zfilipin@deploy1001> |
Synchronized wmf-config/throttle.php: SWAT: [[gerrit:469168|New throttle rule for Wikipedia in Ort (T207714)]] (duration: 00m 46s) |
[production] |
| 11:11 |
<zfilipin@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:469180|Enable RCPatrol for srwikiquote (T207732)]] (duration: 00m 47s) |
[production] |
| 10:13 |
<ema> |
upload libc++ 6.0.1 to stretch-wikimedia/main T204232 |
[production] |
| 09:42 |
<jynus> |
stopping db1087 to fix db1124 |
[production] |
| 09:31 |
<gehel> |
depooling / banning elastics1017 and 1022 - T207724 |
[production] |
| 09:13 |
<godog> |
roll-restart thumbor to send statsd traffic through statsd_exporter - T205870 |
[production] |
| 08:08 |
<godog> |
update hp firmware to 6.60 on ms-be2017 - T141756 |
[production] |
| 07:14 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1100 - T184805 (duration: 00m 48s) |
[production] |
| 06:50 |
<elukey> |
powercycle ms-be2017 (frozen since ~8hrs ago) |
[production] |
| 06:42 |
<elukey> |
restart yarn and hdfs daemon on analytics1068 to pick up correct config (the host was down since before we swapped the Hadoop masters due to hw failure) |
[production] |
| 06:39 |
<marostegui> |
Stop replication on db1092 and db1087 for checking T206743 |
[production] |
| 06:02 |
<marostegui> |
Deploy schema change on s3 - T207359 |
[production] |
| 00:35 |
<SMalyshev> |
temp depooled wdq1003 to let it catch up |
[production] |
| 00:17 |
<Amir1> |
evening SWAT is done |
[production] |
|
2018-10-22
§
|
| 23:59 |
<ladsgroup@deploy1001> |
Synchronized php-1.32.0-wmf.26/includes/changetags/ChangeTags.php: SWAT: [[gerrit:469114|Fix bad join on ChangeTag subquery (T207313)]] (duration: 00m 47s) |
[production] |
| 23:39 |
<smalyshev@deploy1001> |
Finished deploy [wdqs/wdqs@d4692ea]: Redeploy Updater for T207673 (duration: 10m 12s) |
[production] |
| 23:29 |
<smalyshev@deploy1001> |
Started deploy [wdqs/wdqs@d4692ea]: Redeploy Updater for T207673 |
[production] |
| 22:12 |
<pmiazga@deploy1001> |
Synchronized wmf-config//InitialiseSettings-labs.php: SWAT: [[gerrit:469121|beta: Disable page issues A/B test on beta cluster only (T200792)]] (duration: 00m 46s) |
[production] |
| 21:44 |
<mutante> |
adding new prod ServerAlias punjabi.wikimedia.org to Apache cluster (T207583) |
[production] |
| 21:13 |
<ayounsi@deploy1001> |
Finished deploy [librenms/librenms@0fd8da6]: Revert LibreNMS upgrade - T207481 (duration: 00m 08s) |
[production] |
| 21:13 |
<ayounsi@deploy1001> |
Started deploy [librenms/librenms@0fd8da6]: Revert LibreNMS upgrade - T207481 |
[production] |
| 21:08 |
<andrewbogott> |
rebooting cloudvirt1023 |
[production] |
| 20:52 |
<ayounsi@deploy1001> |
Finished deploy [librenms/librenms@737683a]: Upgreade LibreNMS to 1.44 - T207481 (duration: 00m 10s) |
[production] |
| 20:52 |
<ayounsi@deploy1001> |
Started deploy [librenms/librenms@737683a]: Upgreade LibreNMS to 1.44 - T207481 |
[production] |
| 20:28 |
<ladsgroup@deploy1001> |
Finished deploy [ores/deploy@e89e880]: Use redis task tracker (T152012) (duration: 22m 02s) |
[production] |
| 20:06 |
<ladsgroup@deploy1001> |
Started deploy [ores/deploy@e89e880]: Use redis task tracker (T152012) |
[production] |
| 18:54 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT Deploy TemplateWizard everywhere T202545, re-try (duration: 00m 45s) |
[production] |
| 18:50 |
<jforrester@deploy1001> |
scap failed: average error rate on 4/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) |
[production] |
| 18:48 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: [Beta] Temporarily disable WBMI from Beta Commons whilst Wikibse is fixed T180981 (duration: 00m 46s) |
[production] |
| 18:38 |
<jforrester@deploy1001> |
Synchronized php-1.32.0-wmf.26/resources/src/mediawiki.rcfilters/styles/mw.rcfilters.ui.ChangesListWrapperWidget.highlightCircles.seenunseen.less: SWAT RCFIlters: Fix highlight circles for unseen changes T207472 (duration: 00m 46s) |
[production] |
| 18:36 |
<jforrester@deploy1001> |
Synchronized php-1.32.0-wmf.26/skins/MinervaNeue/resources/skins.minerva.scripts/pageIssuesLogger.js: SWAT Fix reading depth logging part 2 T207423 (duration: 00m 46s) |
[production] |
| 18:35 |
<jforrester@deploy1001> |
Synchronized php-1.32.0-wmf.26/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.readingDepth.js: SWAT Fix reading depth logging part 1 T207423 (duration: 00m 46s) |
[production] |
| 18:31 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT Add TemplateWizard to the BF allow list T205290 (duration: 00m 48s) |
[production] |
| 18:05 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: [Beta Cluster] Load but don't enable MediaInfo on Beta Commons cf. T180981 (duration: 00m 45s) |
[production] |
| 18:00 |
<jforrester@deploy1001> |
Synchronized wmf-config/Wikibase.php: For WikibaseMediaInfo wikis, load basic Wikibase repo code cf. T180981 (duration: 00m 46s) |
[production] |
| 17:56 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Allow enablement of the WikibaseMediaInfo, still off everywhere cf. T180981 (duration: 00m 48s) |
[production] |
| 17:46 |
<jforrester@deploy1001> |
Synchronized wmf-config/extension-list: Add WikibaseMediaInfo i18n to cache cf. T180981 (duration: 00m 46s) |
[production] |
| 17:40 |
<mobrovac@deploy1001> |
Finished deploy [proton/deploy@b3e254a]: Update Puppeteer to v1.9.0 - T207416 (duration: 01m 34s) |
[production] |
| 17:38 |
<mobrovac@deploy1001> |
Started deploy [proton/deploy@b3e254a]: Update Puppeteer to v1.9.0 - T207416 |
[production] |
| 17:34 |
<onimisionipe@deploy1001> |
Finished deploy [wdqs/wdqs@975a67b]: WDQS deployment - GUI update and binaries upgrade (duration: 11m 47s) |
[production] |
| 17:23 |
<XioNoX> |
enable cr2:xe-4/0/0 (to asw-a) for optics replacement - T203719 |
[production] |
| 17:22 |
<onimisionipe@deploy1001> |
Started deploy [wdqs/wdqs@975a67b]: WDQS deployment - GUI update and binaries upgrade |
[production] |
| 17:20 |
<XioNoX> |
disable cr2:xe-4/0/0 (to asw-a) for optics replacement - T203719 |
[production] |
| 17:19 |
<elukey@deploy1001> |
Finished deploy [analytics/refinery@1de5f44]: Deploy new version of Camus and pageview whitelist (duration: 07m 05s) |
[production] |
| 17:16 |
<cmjohnson1> |
analytics1068 down for mother board swap |
[production] |
| 17:12 |
<elukey@deploy1001> |
Started deploy [analytics/refinery@1de5f44]: Deploy new version of Camus and pageview whitelist |
[production] |
| 17:11 |
<XioNoX> |
re-enable puppet fleet-wide for puppetmaster1001 uplink move |
[production] |
| 17:10 |
<XioNoX> |
moving puppetmaster1001 uplink to asw2-b |
[production] |
| 17:08 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@431a55d]: Rolling out fix for T207510 (duration: 03m 40s) |
[production] |