|
2019-07-16
§
|
| 11:12 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 11:12 |
<moritzm> |
rebooting remaining swift frontends in eqiad to pick up a kernel with SACK fixed (T228086) |
[production] |
| 10:29 |
<moritzm> |
rebooting ms-fe1005 to pick up kernel with SACK fixed (T228086) |
[production] |
| 10:28 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 10:28 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 10:17 |
<vgutierrez> |
restart pybal on lvs1013 |
[production] |
| 10:15 |
<vgutierrez> |
restart pybal on lvs2001 |
[production] |
| 10:11 |
<vgutierrez> |
restarting pybal on lvs1016 |
[production] |
| 10:08 |
<vgutierrez> |
restarting pybal on lvs2004 |
[production] |
| 10:04 |
<vgutierrez@puppetmaster1001> |
conftool action : set/pooled=yes; selector: cluster=ncredir,service=nginx |
[production] |
| 09:24 |
<elukey> |
apply mcrouter async replication settings to mw1276 - T225642 |
[production] |
| 09:23 |
<elukey> |
pool mw1261 back with mcrouter async replication settings - T225642 |
[production] |
| 08:50 |
<fsero> |
upload coredns docker image into registry T226516 |
[production] |
| 08:44 |
<jynus> |
droping servermon accounts from m1 dbs T198939 |
[production] |
| 08:12 |
<fsero> |
uploading coredns_1.5.2 for buster and stretch - T226516 |
[production] |
| 08:11 |
<fsero> |
uploading coredns_1.5.2 for buster and stretch |
[production] |
| 07:45 |
<elukey> |
depool mw1261 to test mcrouter changes |
[production] |
| 00:24 |
<krinkle@deploy1001> |
Synchronized php-1.34.0-wmf.13/includes/cache/LinkCache.php: 4a5f4ca2fd788 (duration: 00m 51s) |
[production] |
| 00:05 |
<catrope@deploy1001> |
Synchronized php-1.34.0-wmf.13/skins/MinervaNeue/: Restrict AMC scripts and styles to AMC mode (T227929) (duration: 00m 52s) |
[production] |
| 00:03 |
<shdubsh> |
restart logstash to revert mitigations - T228089 |
[production] |
|
2019-07-15
§
|
| 23:55 |
<XioNoX> |
rotate network-root password |
[production] |
| 23:31 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 23:31 |
<robh@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 23:07 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: GrowthExperiments: Remove reference to non-existent feature flag (duration: 00m 51s) |
[production] |
| 22:33 |
<XenoRyet> |
updated civicrm from 8a4451f390 to 3be1a8c77c |
[production] |
| 22:01 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Stop setting wgNonincludableNamespaces, default, never varied (duration: 00m 52s) |
[production] |
| 22:00 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Drop wmgEnableTabularData and wmgEnableMapData, unused (duration: 00m 55s) |
[production] |
| 21:58 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Use wmgEnableJsonConfigDataMode instead of wmgEnableTabularData and wmgEnableMapData (duration: 00m 56s) |
[production] |
| 21:55 |
<jijiki> |
Depool mw1239 for maintenance - T227867 |
[production] |
| 21:55 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Add wmgEnableJsonConfigDataMode to IS (duration: 00m 55s) |
[production] |
| 21:46 |
<sbassett@deploy1001> |
Synchronized private/PrivateSettings.php: Add more severe rate limits for eswikiquote (T227416) (duration: 00m 50s) |
[production] |
| 21:16 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 21:06 |
<XioNoX> |
rollback `as-path HE ".* 6939 .*"` to AVOID-PATH in eqsin - T228015 |
[production] |
| 20:59 |
<krinkle@deploy1001> |
Synchronized php-1.34.0-wmf.13/includes/Title.php: T227700 / T227700: getSubpage should not lose the interwiki prefix (duration: 00m 52s) |
[production] |
| 20:54 |
<bsitzmann@deploy1001> |
Finished deploy [mobileapps/deploy@bc3a2fd]: Update mobileapps to 7fd39da (T227907) (duration: 02m 24s) |
[production] |
| 20:52 |
<bsitzmann@deploy1001> |
Started deploy [mobileapps/deploy@bc3a2fd]: Update mobileapps to 7fd39da (T227907) |
[production] |
| 20:52 |
<bsitzmann@deploy1001> |
Finished deploy [mobileapps/deploy@bc3a2fd]: Update mobileapps to 7fd39da (T227907) (duration: 07m 53s) |
[production] |
| 20:50 |
<Krinkle> |
deploy1001: Unable to fetch git commits from Gerrit for php-1.34.0-wmf.13 due to "error: cannot update the ref 'refs/remotes/origin/fundraising/REL1_31': unable to append to '.git/logs/refs/remotes/origin/fundraising/REL1_31': Permission denied" |
[production] |
| 20:47 |
<XioNoX> |
add `as-path HE ".* 6939 .*"` to AVOID-PATH in eqsin - T228015 |
[production] |
| 20:44 |
<bsitzmann@deploy1001> |
Started deploy [mobileapps/deploy@bc3a2fd]: Update mobileapps to 7fd39da (T227907) |
[production] |
| 20:30 |
<XioNoX> |
deactivate HE peering in eqsin - T228015 |
[production] |
| 20:02 |
<jynus> |
reducing consistency of db2045 to avoid lag at T227862 |
[production] |
| 19:53 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 19:31 |
<ppchelko@deploy1001> |
Finished deploy [cpjobqueue/deploy@fd0a41a]: Change the name of the error log field for deduplicatio (duration: 01m 13s) |
[production] |
| 19:30 |
<ppchelko@deploy1001> |
Started deploy [cpjobqueue/deploy@fd0a41a]: Change the name of the error log field for deduplicatio |
[production] |
| 19:27 |
<ppchelko@deploy1001> |
Finished deploy [changeprop/deploy@df6322a]: Rename error field in deduplication logs (duration: 01m 28s) |
[production] |
| 19:26 |
<ppchelko@deploy1001> |
Started deploy [changeprop/deploy@df6322a]: Rename error field in deduplication logs |
[production] |
| 19:25 |
<XenoRyet> |
update payments-wiki from 59ace50d66 to 224c6b2d7b |
[production] |
| 19:10 |
<thcipriani> |
gerrit back |
[production] |
| 19:09 |
<thcipriani> |
gerrit restart for v2.15.14 |
[production] |