|
2020-06-30
§
|
| 21:40 |
<crusnov@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 21:38 |
<crusnov@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 21:38 |
<crusnov@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) |
[production] |
| 21:38 |
<crusnov@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 19:19 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: group 0 wikis to 1.35.0-wmf.39 # T254176 |
[production] |
| 18:31 |
<cdanis> |
T256790 ✔️ cdanis@netflow2001.codfw.wmnet ~ 🕝☕ sudo apt install valgrind |
[production] |
| 18:27 |
<tgr> |
Morning deploys done |
[production] |
| 18:23 |
<tgr@deploy1001> |
Synchronized php-1.35.0-wmf.39/extensions/ElectronPdfService/src/ElectronPdfServiceHooks.php: Backport: [[gerrit:608485|Hotfix: "Undefined index: print" (T256761)]] (duration: 01m 05s) |
[production] |
| 18:11 |
<shdubsh> |
restart varnishmtail,atsmtail,ncredirmtail on ncredir,cp hosts in codfw and eqsin |
[production] |
| 18:05 |
<cdanis> |
installing libc6-dbg on netflow2001 T256790 |
[production] |
| 17:40 |
<mdholloway> |
mobileapps deployments on k8s failing with timeouts; filed T256786 |
[production] |
| 17:37 |
<cdanis> |
✔️ cdanis@netflow2001.codfw.wmnet ~ 🕜☕ sudo systemctl restart nfacctd |
[production] |
| 17:33 |
<mholloway-shell@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . |
[production] |
| 17:18 |
<mholloway-shell@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . |
[production] |
| 17:17 |
<papaul> |
uplugging msw-c3 power to relocate port on PDU |
[production] |
| 17:09 |
<mholloway-shell@deploy1001> |
Finished deploy [mobileapps/deploy@f9df1af]: Update mobileapps to 5c7611b9 (duration: 03m 33s) |
[production] |
| 17:05 |
<mholloway-shell@deploy1001> |
Started deploy [mobileapps/deploy@f9df1af]: Update mobileapps to 5c7611b9 |
[production] |
| 16:57 |
<cdanis> |
T256444 restarted purged on cp2030 and repooling |
[production] |
| 16:48 |
<cdanis> |
T256444 ✔️ cdanis@cp2030.codfw.wmnet ~ 🕐☕ sudo depool |
[production] |
| 15:54 |
<otto@deploy1001> |
Finished deploy [analytics/refinery@d63944e]: Deploying new camus wmf10 jar to an-launcher1002 for T256370 - take 3 (duration: 00m 03s) |
[production] |
| 15:54 |
<otto@deploy1001> |
Started deploy [analytics/refinery@d63944e]: Deploying new camus wmf10 jar to an-launcher1002 for T256370 - take 3 |
[production] |
| 15:40 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:35 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:32 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:30 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:24 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:20 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:16 |
<otto@deploy1001> |
Finished deploy [analytics/refinery@1112749]: roll back to 1112749 on an-launcher1002, git-fat not pulling artifacts (duration: 01m 21s) |
[production] |
| 15:14 |
<otto@deploy1001> |
Started deploy [analytics/refinery@1112749]: roll back to 1112749 on an-launcher1002, git-fat not pulling artifacts |
[production] |
| 15:14 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:10 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:10 |
<moritzm> |
rebooting mwdebug* hosts for kernel security update |
[production] |
| 15:05 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 15:03 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 15:01 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 14:59 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 14:59 |
<moritzm> |
rebooting failoid hosts for kernel update |
[production] |
| 14:49 |
<otto@deploy1001> |
Finished deploy [analytics/refinery@d63944e]: Deploying new camus wmf10 jar to an-launcher1002 for T256370 - take 3 (duration: 00m 03s) |
[production] |
| 14:49 |
<otto@deploy1001> |
Started deploy [analytics/refinery@d63944e]: Deploying new camus wmf10 jar to an-launcher1002 for T256370 - take 3 |
[production] |
| 14:47 |
<otto@deploy1001> |
Finished deploy [analytics/refinery@d63944e]: Deploying new camus wmf10 jar to an-launcher1002 for T256370 - take 2 (duration: 00m 03s) |
[production] |
| 14:47 |
<otto@deploy1001> |
Started deploy [analytics/refinery@d63944e]: Deploying new camus wmf10 jar to an-launcher1002 for T256370 - take 2 |
[production] |
| 14:43 |
<hashar> |
Train blocked on Flow being broken: T256761 # T254176 |
[production] |
| 14:38 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: Revert "group0 wikis to 1.35.0-wmf.39" - T256759 |
[production] |
| 14:34 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 14:32 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 14:28 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 14:26 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 14:25 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.35.0-wmf.39 |
[production] |
| 14:21 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:21 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |