|
2018-07-31
ยง
|
| 16:25 |
<elukey> |
pool eventbus on kafka1002 after network maintenance |
[production] |
| 16:24 |
<ayounsi@puppetmaster1001> |
conftool action : set/pooled=yes; selector: dc=eqiad,cluster=aqs,service=cassandra,name=aqs1004.eqiad.wmnet |
[production] |
| 16:24 |
<ayounsi@puppetmaster1001> |
conftool action : set/pooled=yes; selector: dc=eqiad,cluster=aqs,service=aqs,name=aqs1004.eqiad.wmnet |
[production] |
| 16:16 |
<elukey> |
precautionary restart of eventbus on kafka1002 after network downtime (DNS name res errors, Kafka broker conn issues, etc..) |
[production] |
| 16:07 |
<marostegui> |
Reload dbproxy1003 and dbproxy1008 |
[production] |
| 16:06 |
<ema> |
reboot cp3044 for kernel update |
[production] |
| 16:01 |
<ayounsi@puppetmaster1001> |
conftool action : set/pooled=yes; selector: dc=eqiad,cluster=prometheus,service=prometheus,name=prometheus1004.eqiad.wmnet |
[production] |
| 15:51 |
<ayounsi@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=druid-public,service=druid-public-broker,name=druid1005.eqiad.wmnet |
[production] |
| 15:50 |
<ayounsi@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=aqs,service=cassandra,name=aqs1004.eqiad.wmnet |
[production] |
| 15:50 |
<ayounsi@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=aqs,service=aqs,name=aqs1004.eqiad.wmnet |
[production] |
| 15:46 |
<ayounsi@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=eventbus,service=eventbus,name=kafka1002.eqiad.wmnet |
[production] |
| 15:45 |
<ayounsi@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=prometheus,service=prometheus,name=prometheus1004.eqiad.wmnet |
[production] |
| 15:36 |
<ema> |
power-cycle cp3031, stuck rebooting |
[production] |
| 15:29 |
<_joe_> |
restarting apache on phab1001 |
[production] |
| 15:20 |
<godog> |
depool thumbor100[12] ahead of switch move - T183585 |
[production] |
| 15:01 |
<XioNoX> |
starting the eqiad row B servers move - T183585 |
[production] |
| 13:51 |
<moritzm> |
installing ffmpeg 3.2.12 security updates on video scalers |
[production] |
| 13:34 |
<akosiaris> |
repool mathoid eqiad cluster in discovery |
[production] |
| 13:18 |
<akosiaris> |
upgrade kubernetes1004 to 1.9.9 |
[production] |
| 13:14 |
<akosiaris> |
upgrade kubernetes1002 to 1.9.9 |
[production] |
| 13:14 |
<akosiaris> |
upgrade kubernetes1003 to 1.9.9 |
[production] |
| 13:03 |
<akosiaris> |
upgrade kubernetes1001 to 1.9.9 |
[production] |
| 12:58 |
<akosiaris> |
upgrade chlorine.eqiad.wmnet to kubernetes 1.9.9 |
[production] |
| 12:54 |
<akosiaris> |
upgrade argon.eqiad.wmnet to kubernetes 1.9.9 |
[production] |
| 12:45 |
<akosiaris> |
depool eqiad mathoid in discovery |
[production] |
| 12:29 |
<gehel@puppetmaster1001> |
conftool action : set/weight=15; selector: dc=eqiad,cluster=wdqs,name=wdqs1005.eqiad.wmnet |
[production] |
| 12:29 |
<gehel@puppetmaster1001> |
conftool action : set/weight=15; selector: dc=eqiad,cluster=wdqs,name=wdqs1004.eqiad.wmnet |
[production] |
| 12:29 |
<gehel> |
rebalance LVS weights to send less traffic to wdqs1003 - T200563 |
[production] |
| 12:13 |
<reedy@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Remove logging now in mw core (duration: 00m 48s) |
[production] |
| 12:11 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: wmg - for Sentry (duration: 00m 48s) |
[production] |
| 12:09 |
<reedy@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Remove wmg - for Sentry (duration: 00m 48s) |
[production] |
| 12:06 |
<reedy@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Load Sentry with wfLoadExtension (duration: 00m 48s) |
[production] |
| 12:05 |
<reedy@deploy1001> |
Synchronized wmf-config/liquidthreads.php: Remove lqt cruft, load with wfLoadExtension (duration: 00m 48s) |
[production] |
| 11:44 |
<gehel> |
restarting blazegraph on wdqs1003 - JVM unresponsive |
[production] |
| 11:38 |
<zeljkof> |
EU SWAT finished |
[production] |
| 11:37 |
<zfilipin@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:448050|Introduce autopatrolled on bnwikisource (T199475)]] (duration: 00m 48s) |
[production] |
| 11:30 |
<zfilipin@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:449391|Use HD logos for hewiktionary (T200470)]] (duration: 00m 48s) |
[production] |
| 11:24 |
<zfilipin@deploy1001> |
Synchronized static/images/project-logos/: SWAT: [[gerrit:449390|Upload HD logos for hewiktionary (T200470)]] (duration: 00m 48s) |
[production] |
| 11:16 |
<zfilipin@deploy1001> |
Synchronized static/images/project-logos: SWAT: [[gerrit:449389|Milestone logo for atjwiki (T200713)]] (duration: 00m 49s) |
[production] |
| 11:10 |
<zfilipin@deploy1001> |
Synchronized static/images/project-logos/hewikiquote.png: SWAT: [[gerrit:448154|Update hewikiquote logo (T200296)]] (duration: 00m 50s) |
[production] |
| 10:55 |
<_joe_> |
test log |
[production] |
| 10:40 |
<akosiaris@deploy1001> |
scap-helm mathoid finished |
[production] |
| 10:40 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster codfw completed |
[production] |
| 10:39 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster eqiad completed |
[production] |
| 10:39 |
<akosiaris@deploy1001> |
scap-helm mathoid upgrade production stable/mathoid [namespace: mathoid, clusters: eqiad,codfw] |
[production] |
| 09:46 |
<ema> |
reboot cp1045 (jessie) for kernel updates |
[production] |
| 09:29 |
<ema> |
upload varnishkafka 1.0.13-1 to stretch-wikimedia T200445 T186250 |
[production] |
| 08:44 |
<jynus> |
stop and reimage db1104 to stretch |
[production] |
| 08:17 |
<akosiaris> |
repool mathoid codfw in discovery, the kubernetes upgrade is done |
[production] |
| 08:08 |
<akosiaris> |
upgrade kubernetes2004 to 1.9.9 |
[production] |