|
2019-11-05
§
|
| 11:57 |
<aborrero@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 11:57 |
<aborrero@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 10:59 |
<ema> |
pool cp5012 with ATS backend T227432 |
[production] |
| 10:45 |
<vgutierrez> |
restarting atsmtail@backend on cp5006 |
[production] |
| 09:36 |
<ema@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:34 |
<ema@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:24 |
<ema> |
wb2-phab stopped saying things a while ago. Restarted |
[production] |
| 09:18 |
<jynus> |
restart dbprov100[12] T236924 |
[production] |
| 09:11 |
<jynus> |
restart dbprov2001 T236924 |
[production] |
| 08:12 |
<vgutierrez> |
uploaded fifo-log-demux 0.6 to apt.wikimedia.org (stretch) |
[production] |
| 08:02 |
<jynus> |
redact mnwwiki on db1124 and db2094 T235743 |
[production] |
| 04:30 |
<vgutierrez> |
Switch from nginx to ats-tls on cp5011 - T231627 |
[production] |
| 04:13 |
<vgutierrez> |
Switch from nginx to ats-tls on cp5010 - T231627 |
[production] |
| 03:51 |
<vgutierrez> |
pooling cp3057 - T237348 |
[production] |
| 03:46 |
<mutante> |
wdqs1004 restarting wdqs-blazegraph |
[production] |
| 03:01 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=cp3057.esams.wmnet |
[production] |
| 02:59 |
<vgutierrez> |
depool cp3057 - T237348 |
[production] |
| 00:15 |
<mutante> |
gerrit - restarting service to re-enable jgit gc (T217497) |
[production] |
| 00:13 |
<mutante> |
gerrit2001 - restart gerrit (replica) |
[production] |
|
2019-11-04
§
|
| 23:18 |
<milimetric@deploy1001> |
Finished deploy [analytics/refinery@99f1535]: Fix for geoeditors jobs (duration: 07m 20s) |
[production] |
| 23:11 |
<milimetric@deploy1001> |
Started deploy [analytics/refinery@99f1535]: Fix for geoeditors jobs |
[production] |
| 23:05 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 23:03 |
<gehel@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:08 |
<bd808> |
The Wikimedia SAL Twitter feed is now @wikimedia_sal (https://twitter.com/wikimedia_sal) T237322 |
[production] |
| 20:51 |
<bd808> |
Testing twitter feed following account confirmation |
[production] |
| 19:23 |
<Urbanecm> |
Morning SWAT done |
[production] |
| 19:17 |
<mutante> |
cobalt - stopping services, removing apache2 |
[production] |
| 19:16 |
<urbanecm@deploy1001> |
Synchronized wmf-config/throttle.php: SWAT: 6a4b966: Add throttle rule for bard college editathon (T236955) (duration: 00m 54s) |
[production] |
| 19:07 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 9204768: Enable DNS blacklist for es.wikinews (T237151) (duration: 00m 53s) |
[production] |
| 19:05 |
<urbanecm@deploy1001> |
Synchronized wmf-config/CommonSettings.php: SWAT: 0fc3909: Allow FlaggedRevs autoreview permission to be assigned globally (duration: 00m 54s) |
[production] |
| 18:30 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@1ac26da]: add new user-selected puppet edit mode (duration: 03m 27s) |
[production] |
| 18:26 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@1ac26da]: add new user-selected puppet edit mode |
[production] |
| 18:24 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@20c710d]: Bump Parsoid-PHP mirroring to 100% T235902 (duration: 14m 30s) |
[production] |
| 18:17 |
<onimisionipe@deploy1001> |
Finished deploy [wdqs/wdqs@2cb2dde]: Event logging via Event Gate and Absolute classpath for munge and runUpdate scripts (duration: 12m 07s) |
[production] |
| 18:09 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@20c710d]: Bump Parsoid-PHP mirroring to 100% T235902 |
[production] |
| 18:05 |
<onimisionipe@deploy1001> |
Started deploy [wdqs/wdqs@2cb2dde]: Event logging via Event Gate and Absolute classpath for munge and runUpdate scripts |
[production] |
| 17:41 |
<jforrester@deploy1001> |
Synchronized multiversion/MWConfigCacheGenerator.php: Update for YAML-reading (offline) (duration: 00m 52s) |
[production] |
| 17:39 |
<jforrester@deploy1001> |
Synchronized wmf-config/config/: Sync out YAML config files (duration: 00m 56s) |
[production] |
| 15:43 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: Enable revert counts on beta (T234955) (duration: 00m 53s) |
[production] |
| 15:36 |
<jynus> |
running failing check_private_data report on labsdb1009 T235743 |
[production] |
| 15:33 |
<ladsgroup@deploy1001> |
Synchronized php-1.35.0-wmf.4/extensions/Wikibase: Wikibase deadlock reduction, [[gerrit:548279|Stop locking and use DISTINCT when finding used terms to delete]] (T236466) (duration: 00m 59s) |
[production] |
| 15:01 |
<joal@deploy1001> |
Started restart [analytics/aqs/deploy@59a97fa]: (no justification provided) |
[production] |
| 14:36 |
<ema@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 14:36 |
<ema@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:53 |
<ema> |
upload trafficserver 8.0.5-1wm10 to stretch-wikimedia |
[production] |
| 13:49 |
<ema@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:47 |
<ema@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:38 |
<elukey> |
update bacula terms on analytics-in{4,6} filters on cr{1,2}-eqiad - T237016 |
[production] |
| 13:28 |
<jbond42> |
update production puppetmasters to use new puppetdb servers |
[production] |
| 13:20 |
<Amir1> |
Creating Mon Wikipedia is done T235739 |
[production] |