|
2019-07-17
ยง
|
| 18:15 |
<mutante> |
mw2181 - sudo: /usr/local/bin/mwscript: command not found on scap pull ?? |
[production] |
| 18:14 |
<mutante> |
mw2181 - scap pull (T205240) |
[production] |
| 18:06 |
<cdanis> |
cdanis@cumin1001.eqiad.wmnet ~ % sudo debdeploy deploy -u 2019-07-17-conftool.yaml -s mw-canary |
[production] |
| 18:02 |
<cdanis> |
upgrade to python3-conftool 1.1.1-1 on mwdebug2001 |
[production] |
| 18:01 |
<cdanis> |
cdanis@install1002.wikimedia.org ~ % sudo -E reprepro -C main include jessie-wikimedia conftool/conftool_1.1.1-1+deb8u1_amd64.changes |
[production] |
| 18:01 |
<cdanis> |
cdanis@install1002.wikimedia.org ~ % sudo -E reprepro -C main include buster-wikimedia conftool/conftool_1.1.1-1+deb10u1_amd64.changes |
[production] |
| 18:01 |
<cdanis> |
cdanis@install1002.wikimedia.org ~ % sudo -E reprepro -C main include stretch-wikimedia conftool/conftool_1.1.1-1_amd64.changes |
[production] |
| 17:09 |
<papaul> |
shutting down restbase2009 for firmware upgrade |
[production] |
| 17:06 |
<liw@deploy1001> |
rebuilt and synchronized wikiversions files: Revert "group[0|1] wikis to 1.34.0-wmf.13" |
[production] |
| 16:57 |
<dcausse> |
morning swat done |
[production] |
| 16:54 |
<dcausse@deploy1001> |
Synchronized php-1.34.0-wmf.14/extensions/CirrusSearch/includes/ElasticaErrorHandler.php: T228283: Log response data JSON on errors (duration: 00m 55s) |
[production] |
| 16:48 |
<Urbanecm> |
Deployed patch for T207094 |
[production] |
| 16:47 |
<gehel@cumin1001> |
END (FAIL) - Cookbook sre.postgresql.postgres-init (exit_code=99) |
[production] |
| 16:40 |
<elukey> |
execute reprepro clearvanished on install1002 to clear buster-wikimedia|thirdparty/amd-rocm (not used anymore) |
[production] |
| 16:37 |
<dcausse> |
reponing morning SWAT |
[production] |
| 16:24 |
<papaul> |
shutting down mw2181 for firmware upgrade |
[production] |
| 16:20 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:20 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:19 |
<jijiki> |
Depool mw2181 - T205240 |
[production] |
| 16:08 |
<Urbanecm> |
Morning SWAT done |
[production] |
| 16:07 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Raise zh_classicalwiki requirement for autoconfirmed (T228141) (duration: 00m 55s) |
[production] |
| 16:07 |
<cmjohnson1> |
powering off cloudvirt1014 for rack move T226188 |
[production] |
| 16:05 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:523686|Enable partial blocks on dewiki]] (T228150) (duration: 00m 54s) |
[production] |
| 16:01 |
<jbond42> |
copy confd package from stretch-wikimedia to buster-wikimedia |
[production] |
| 15:46 |
<Urbanecm> |
Re-syncing patch for T207094 T228284 and wmf.14 |
[production] |
| 15:37 |
<Urbanecm> |
Deployed patch for T207094 T228284 to wmf.13 and wmf.14 |
[production] |
| 15:15 |
<fsero> |
restarting swift-container-sync on ms-be* for getting logging configuration T228196 |
[production] |
| 15:11 |
<papaul> |
shutting down mw2250 for disk replacement |
[production] |
| 15:10 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 15:07 |
<hashar> |
upgrading CI Jenkins # T228142 |
[production] |
| 15:06 |
<papaul> |
shutting down ms-be2022 for HW troubleshooting |
[production] |
| 15:06 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 15:05 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:03 |
<jijiki> |
Depool mw2269 to reboot it - T227548 |
[production] |
| 15:00 |
<godog> |
poweroff ms-be2022 - T227667 |
[production] |
| 14:55 |
<moritzm> |
updated jenkins in thirdparty/ci (stretch) and thirdparty (jessie) to 2.176.2 (T228142) |
[production] |
| 14:45 |
<fsero> |
enabling container-sync logging T228196 |
[production] |
| 14:41 |
<otto@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 14:41 |
<otto@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 14:35 |
<moritzm> |
restart pybal on lvs2002 (codfw primary) T227778 |
[production] |
| 14:32 |
<gehel@cumin1001> |
START - Cookbook sre.postgresql.postgres-init |
[production] |
| 14:30 |
<gehel> |
repool maps1004 - T218097 |
[production] |
| 14:11 |
<liw@deploy1001> |
Synchronized php: group1 wikis to 1.34.0-wmf.14 (duration: 00m 54s) |
[production] |
| 14:10 |
<liw@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.34.0-wmf.14 |
[production] |
| 14:09 |
<moritzm> |
restarting pybal on backup LVSes in codfw |
[production] |
| 14:02 |
<liw@deploy1001> |
Synchronized php-1.34.0-wmf.14/extensions/CirrusSearch/includes/Searcher.php: Do not serialize ResultsType instance T228276 (duration: 00m 55s) |
[production] |
| 13:37 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 13:26 |
<moritzm> |
disabled puppet on Icinga hosts in preparation of adding the LDAP replicas/codfw to LVS |
[production] |
| 13:10 |
<ema> |
cp-codfw: varnish frontend rolling restarts for 5.1.3-1wm11 upgrades T227672 |
[production] |
| 13:06 |
<ema> |
prometheus servers: remove varnish-upload_$dc_backend.yaml, replaced by ATS equivalent T227668 |
[production] |