|
2017-12-07
ยง
|
| 17:33 |
<demon@tin> |
Synchronized php-1.31.0-wmf.11/extensions/VisualEditor/lib/ve: Ief480487, Deskana made me do it (duration: 00m 49s) |
[production] |
| 17:25 |
<milimetric@tin> |
Finished deploy [analytics/aqs/deploy@4ec13b4]: (no justification provided) (duration: 07m 28s) |
[production] |
| 17:25 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mw1314.eqiad.wmnet |
[production] |
| 17:25 |
<herron> |
puppetmaster1001 upgraded to puppet 4. re-enabling puppet agents across the fleet |
[production] |
| 17:18 |
<milimetric@tin> |
Started deploy [analytics/aqs/deploy@4ec13b4]: (no justification provided) |
[production] |
| 17:13 |
<herron> |
upgrading puppetmaster1001 to puppet 4 |
[production] |
| 17:08 |
<herron> |
temporarily disabling all puppet agents during puppetmaster1001 (puppet ca) upgrade to puppet 4 |
[production] |
| 16:14 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1099:3311 weight - T178359 (duration: 00m 48s) |
[production] |
| 16:13 |
<herron> |
upgrading puppetmaster1002 to puppet 4 |
[production] |
| 16:03 |
<moritzm> |
uploaded openssl 1.0.2n for jessie-wikimedia to apt.wikimedia.org |
[production] |
| 16:01 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1093 back as main traffic in s6 - T178359 (duration: 00m 48s) |
[production] |
| 15:44 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1099:3311 weight - T178359 (duration: 00m 48s) |
[production] |
| 15:42 |
<elukey> |
hhvm-dump-debug for mw1314 saved to /tmp/hhvm.17991.bt. |
[production] |
| 15:30 |
<elukey@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=mw1314.eqiad.wmnet |
[production] |
| 15:24 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase db1099:3311 weight - T178359 (duration: 00m 48s) |
[production] |
| 15:19 |
<otto@tin> |
Finished deploy [analytics/superset/deploy@f0f5adf]: initial deployment (duration: 00m 02s) |
[production] |
| 15:18 |
<otto@tin> |
Started deploy [analytics/superset/deploy@f0f5adf]: initial deployment |
[production] |
| 14:55 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db1099:331 with low weight - T178359 (duration: 00m 47s) |
[production] |
| 14:29 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1074 (duration: 00m 47s) |
[production] |
| 14:09 |
<zeljkof> |
EU SWAT finished |
[production] |
| 14:08 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:395835|alswiki: Set wgRestrictDisplayTitle = false (T182154)]] (duration: 00m 49s) |
[production] |
| 13:39 |
<moritzm> |
reimaging mw2246 to stretch |
[production] |
| 12:54 |
<jmm@puppetmaster1001> |
conftool action : set/pooled=yes; selector: mw2152.codfw.wmnet |
[production] |
| 12:32 |
<addshore@tin> |
Synchronized wmf-config/Wikibase.php: [[gerrit:395966|Move Wikibase dispatchingLockManager to InitialiseSettings]] PT 2/2 (duration: 00m 48s) |
[production] |
| 12:31 |
<addshore@tin> |
Synchronized wmf-config/InitialiseSettings.php: [[gerrit:395966|Move Wikibase dispatchingLockManager to InitialiseSettings]] PT 1/2 (duration: 00m 48s) |
[production] |
| 11:42 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase API traffic for db1074 (duration: 00m 48s) |
[production] |
| 11:41 |
<moritzm> |
reimaging mw2152 to stretch |
[production] |
| 11:35 |
<marostegui> |
Compress s8 on db1099 - T178359 |
[production] |
| 11:21 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase API traffic for db1074 (duration: 00m 48s) |
[production] |
| 11:10 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@097ba7d]: Add CORS headers to erroneous responses as well - T182103 (duration: 05m 24s) |
[production] |
| 11:05 |
<mobrovac@tin> |
Started deploy [restbase/deploy@097ba7d]: Add CORS headers to erroneous responses as well - T182103 |
[production] |
| 10:57 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1074 with low weight (duration: 00m 48s) |
[production] |
| 10:50 |
<elukey> |
powercycle analytics1003 - no serial console, ssh stuck in System is booting up. See pam_nologin(8) |
[production] |
| 10:28 |
<_joe_> |
depooling mw1283 for further investigation |
[production] |
| 10:14 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Fully pool db1098:3316 db1098:3317 - T178359 (duration: 00m 51s) |
[production] |
| 10:12 |
<elukey> |
reboot analytics1003 for kernel+jvm updates - T179943 |
[production] |
| 10:01 |
<gehel> |
upgrade of ELK stack on logstash100* completed - Kibana was unavailable for longer than expected - T178412 |
[production] |
| 09:48 |
<hashar> |
CI: removed Wikidata from configuration, replaced by Wikibase. wmf/* and REL branches are going to be broken though | https://gerrit.wikimedia.org/r/395704 | T181838 |
[production] |
| 09:46 |
<akosiaris> |
silence ganeti1006 on icinga T181121 |
[production] |
| 09:36 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase traffic for db1098:3316 db1098:3317 - T178359 (duration: 00m 52s) |
[production] |
| 09:28 |
<marostegui> |
Upgrade MySQL and kernel on db1074 |
[production] |
| 09:22 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=logstash1009.eqiad.wmnet |
[production] |
| 09:19 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=logstash1009.eqiad.wmnet |
[production] |
| 09:18 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=logstash1008.eqiad.wmnet |
[production] |
| 09:14 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=logstash1008.eqiad.wmnet |
[production] |
| 09:13 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=logstash1007.eqiad.wmnet |
[production] |
| 09:08 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=logstash1007.eqiad.wmnet |
[production] |
| 09:05 |
<gehel@tin> |
Finished deploy [logstash/plugins@b13d2fa]: (no justification provided) (duration: 00m 02s) |
[production] |
| 09:05 |
<gehel@tin> |
Started deploy [logstash/plugins@b13d2fa]: (no justification provided) |
[production] |
| 09:00 |
<gehel> |
upgrading ELK stack on logstash100* - some log messages might be lost during the upgrade - T178412 |
[production] |