|
2019-11-20
ยง
|
| 21:01 |
<ssastry@deploy1001> |
Finished deploy [parsoid/deploy@7665624]: Dummy Parsoid deploy to test T238748 fix (duration: 07m 20s) |
[production] |
| 20:53 |
<ssastry@deploy1001> |
Started deploy [parsoid/deploy@7665624]: Dummy Parsoid deploy to test T238748 fix |
[production] |
| 20:37 |
<ssastry@deploy1001> |
Finished deploy [parsoid/deploy@d5646b7]: Updating Parsoid to 2e79460d (duration: 09m 14s) |
[production] |
| 20:27 |
<ssastry@deploy1001> |
Started deploy [parsoid/deploy@d5646b7]: Updating Parsoid to 2e79460d |
[production] |
| 20:27 |
<mholloway-shell@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'wikifeeds' for release 'staging' . |
[production] |
| 20:23 |
<mutante> |
notebook1003 - sudo systemctl nagios-nrpe-server (as usual ....) |
[production] |
| 20:19 |
<mholloway-shell@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'wikifeeds' for release 'staging' . |
[production] |
| 19:31 |
<ejegg> |
updated fundraising internal dashboard from 69fdbec60d to 8fc2726736 |
[production] |
| 19:04 |
<mutante> |
xhgui1001 - initial puppet run, signed puppet cert on puppetmaster1001 |
[production] |
| 18:56 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: RESYNC T221774 - wgWikidataOrgQueryServiceMaxLagFactor 120 (duration: 00m 50s) |
[production] |
| 18:51 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T221774 - wgWikidataOrgQueryServiceMaxLagFactor 120 (duration: 00m 54s) |
[production] |
| 18:42 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T221774 - wgWikidataOrgQueryServiceMaxLagFactor 170 (duration: 00m 53s) |
[production] |
| 18:31 |
<mutante> |
ganeti - introducing and installing buster on new VMs xhgui1001/xhgui2001 - for replacing tungsten (jessie) T238098 |
[production] |
| 18:17 |
<mobrovac> |
morning SWAT done |
[production] |
| 18:17 |
<mobrovac@deploy1001> |
Synchronized php-1.35.0-wmf.5/includes/libs/virtualrest/ParsoidVirtualRESTService.php: Parsoid VRS: Add the Host header - T229015 T229078 T229074 (duration: 00m 52s) |
[production] |
| 18:13 |
<shdubsh> |
restart mtail on fermium |
[production] |
| 17:40 |
<ema> |
pool cp2023 with ATS backend T227432 |
[production] |
| 17:24 |
<mobrovac@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'citoid' for release 'production' . |
[production] |
| 17:21 |
<mobrovac@deploy1001> |
helmfile [CODFW] Ran 'apply' command on namespace 'citoid' for release 'production' . |
[production] |
| 17:19 |
<mobrovac@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'citoid' for release 'staging' . |
[production] |
| 17:18 |
<andrewbogott> |
upgrading pdns to version 4 on cloudservices1003 |
[production] |
| 17:06 |
<ema@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 17:04 |
<ema@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 17:03 |
<andrewbogott> |
upgrading pdns to version 4 on cloudvirt1004 T210715 |
[production] |
| 16:58 |
<andrewbogott> |
disabling puppet on cloudvirt1003 and 1004 for T210715 |
[production] |
| 16:55 |
<moritzm> |
installing rpcbind bugfix updates from buster 10.2 point release |
[production] |
| 16:43 |
<ema@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:40 |
<ema@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:23 |
<ema> |
depool cp2023 and reimage as text_ats T227432 |
[production] |
| 16:14 |
<ema> |
pool cp2019 with ATS backend T227432 |
[production] |
| 16:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1103:3314 after compression', diff saved to https://phabricator.wikimedia.org/P9695 and previous config saved to /var/cache/conftool/dbconfig/20191120-160813-marostegui.json |
[production] |
| 16:03 |
<gehel> |
depool wdqs1004 to allow catching up on lag - T238229 |
[production] |
| 15:42 |
<mobrovac@deploy1001> |
Synchronized wmf-config/LabsServices.php: [BETA-ONLY] Switch Flow to use Parsoid/PHP - T229078 (duration: 00m 52s) |
[production] |
| 15:40 |
<ema@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 15:38 |
<ema@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:36 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: RESYNC T221774 - wgWikidataOrgQueryServiceMaxLagFactor 180 [[gerrit:552069]] (duration: 00m 52s) |
[production] |
| 15:19 |
<ema> |
depool cp2019 and reimage as text_ats T227432 |
[production] |
| 15:08 |
<gehel> |
reset LVS weight for wdqs public eqiad to 10 |
[production] |
| 15:05 |
<effie> |
Enable puppet on mw* |
[production] |
| 14:52 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T221774 - wgWikidataOrgQueryServiceMaxLagFactor 180 [[gerrit:552069]] (duration: 00m 52s) |
[production] |
| 14:50 |
<addshore@deploy1001> |
Synchronized php-1.35.0-wmf.5/extensions/Wikidata.org: T221774 - Wikidata.org extension (use altered lag, not raw lag) [[gerrit:552072]] (duration: 00m 53s) |
[production] |
| 14:49 |
<ema> |
pool cp2016 with ATS backend T227432 |
[production] |
| 14:47 |
<effie> |
disable puppet on all mw* servers |
[production] |
| 14:27 |
<ema@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:24 |
<ema@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:06 |
<ema> |
depool cp2016 and reimage as text_ats T227432 |
[production] |
| 13:32 |
<godog> |
updated puppet compiler facts on compiler100* hosts |
[production] |
| 12:43 |
<ema> |
pool cp2013 with ATS backend T227432 |
[production] |
| 12:27 |
<ema@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:25 |
<ema@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |