|
2019-05-29
ยง
|
| 23:15 |
<mutante> |
repooled phab2001-vcs , fixes pybal / lvs alerts |
[production] |
| 23:13 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=phab2001-vcs.codfw.wmnet |
[production] |
| 23:10 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT Enable wgSpecialSearchFormOptions on production Wikidata T55652 (duration: 00m 57s) |
[production] |
| 23:01 |
<mutante> |
phab2001 - same issue with tin.eqiad.wmnet still showing up when first trying to git clone |
[production] |
| 22:52 |
<mutante> |
misweb2001 - a2dismod mpm_event ; systemctl restart apache2 to fix php7.0 dependency issue |
[production] |
| 22:50 |
<mutante> |
miscweb2001 - when first trying to git pull iegreview - still tries to resolve 'tin.eqiad.wmnet' which is long gone. fix is still to manually edit /srv/deployment/iegreview/iegreview-cache/cache/.git/config |
[production] |
| 22:46 |
<jforrester@deploy1001> |
Synchronized wmf-config/CirrusSearch-common.php: Hot-deploy T224634 to fix CirrusSearch for extension registration (duration: 00m 57s) |
[production] |
| 21:50 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=phab2001-vcs.codfw.wmnet |
[production] |
| 21:47 |
<mutante> |
installing OS on miscweb2001 VM failed at grub install step :( T224323 |
[production] |
| 21:47 |
<mutante> |
sign puppet cert request for phab2001 after reinstall (for some reason it needed me to connect to console and hit enter, reimage script itself was stuck) |
[production] |
| 20:54 |
<mutante> |
creating new ganeti VM miscweb2001.codfw.wmnet with same specs as krypton.eqiad.wmnet (T224323) |
[production] |
| 20:35 |
<arlolra> |
Updated Parsoid to 8546c79 (T219927, T211125) |
[production] |
| 20:35 |
<ejegg> |
updated payments-wiki from 332aaa96e2 to 45b73e7749 |
[production] |
| 20:28 |
<arlolra@deploy1001> |
Finished deploy [parsoid/deploy@6caac43]: Updating Parsoid to 8546c79 (duration: 07m 46s) |
[production] |
| 20:20 |
<arlolra@deploy1001> |
Started deploy [parsoid/deploy@6caac43]: Updating Parsoid to 8546c79 |
[production] |
| 20:10 |
<bblack> |
pool cp3044 (esams cache_upload ats-be) - T222937 |
[production] |
| 19:46 |
<reedy@deploy1001> |
Synchronized php-1.34.0-wmf.7/extensions/Collection/: Replace missing wfCollectionSuggestAction (duration: 00m 57s) |
[production] |
| 19:45 |
<XioNoX> |
enable cr1-codfw:et-0/2/1 - T224511 |
[production] |
| 19:45 |
<reedy@deploy1001> |
Synchronized php-1.34.0-wmf.6/extensions/Collection/: Replace missing wfCollectionSuggestAction (duration: 01m 01s) |
[production] |
| 19:42 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=phab2001-vcs.codfw.wmnet |
[production] |
| 19:32 |
<mutante> |
phba2001 - reinstalling with stretch - upgrade from jessie (T190568) |
[production] |
| 19:09 |
<XioNoX> |
enable cr1-codfw:et-0/2/0 - T224511 |
[production] |
| 18:37 |
<bblack@cumin1001> |
conftool action : set/pooled=no; selector: name=cp3044.esams.wmnet |
[production] |
| 17:44 |
<XioNoX> |
enable cr1-codfw:et-0/0/1 - T224511 |
[production] |
| 17:13 |
<XioNoX> |
enable cr1-codfw:et-0/0/0 - T224511 |
[production] |
| 17:02 |
<urbanecm@deploy1001> |
Synchronized wmf-config/CommonSettings.php: [[:gerrit:501926|Change arwiki default user preferences]], part 3/3 (T220186) (duration: 00m 56s) |
[production] |
| 17:00 |
<urbanecm@deploy1001> |
Synchronized wmf-config/flaggedrevs.php: [[:gerrit:501926|Change arwiki default user preferences]], part 2/3 (T220186) (duration: 00m 56s) |
[production] |
| 16:59 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: [[:gerrit:501926|Change arwiki default user preferences]], part 1/3 (T220186) (duration: 00m 56s) |
[production] |
| 16:48 |
<sbisson@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:512942]] Revert: Hardcode korean help desk config (duration: 00m 56s) |
[production] |
| 16:45 |
<sbisson@deploy1001> |
Synchronized php-1.34.0-wmf.7/extensions/GrowthExperiments/includes/HelpPanel.php: SWAT: [[gerrit:512941]] Prevent parsing of GEHelpPanelHelpDeskTitle from accessing the session (duration: 00m 56s) |
[production] |
| 16:42 |
<sbisson@deploy1001> |
Synchronized php-1.34.0-wmf.6/extensions/GrowthExperiments/includes/HelpPanel.php: SWAT: [[gerrit:512940]] Prevent parsing of GEHelpPanelHelpDeskTitle from accessing the session (duration: 01m 00s) |
[production] |
| 16:32 |
<sbisson@deploy1001> |
Synchronized php-1.34.0-wmf.6/extensions/GrowthExperiments/includes/HelpPanel/QuestionRecord.php: SWAT: [[gerrit:512950]] Revert: Fix phan job: ignore line using JsonSerializable (duration: 00m 57s) |
[production] |
| 16:08 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=0) |
[production] |
| 15:55 |
<jynus> |
upgrade and restart db2087 |
[production] |
| 15:11 |
<moritzm> |
draining ganeti2008 for eventual reboot to pick up MDS-enabled kernel |
[production] |
| 15:06 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 15:06 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:06 |
<anomie@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Setting actor migration to write-new/read-new on group 1 (T188327) (duration: 00m 57s) |
[production] |
| 14:54 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:54 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:54 |
<moritzm> |
draining ganeti2007 for eventual reboot to pick up MDS-enabled kernel |
[production] |
| 14:51 |
<XioNoX> |
`request chassis fpc online slot 0` on cr1-codfw - T224511 |
[production] |
| 14:48 |
<XioNoX> |
`request chassis fpc offline slot 0` on cr1-codfw - T224511 |
[production] |
| 14:47 |
<XioNoX> |
disable et- interfaces on cr1-codfw - T224511 |
[production] |
| 14:45 |
<andrewbogott> |
reimaging cloudcontrol1003 T221770 |
[production] |
| 14:34 |
<moritzm> |
draining ganeti2006 for eventual reboot to pick up MDS-enabled kernel |
[production] |
| 14:34 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:34 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:32 |
<andrewbogott> |
powering off cloudcontrol1003 as one last check to see what explodes before I reimage it |
[production] |
| 14:30 |
<_joe_> |
installing the new service checker on restbase in eqiad |
[production] |