|
2016-02-29
§
|
| 23:56 |
<ebernhardson> |
upgrade elastic2008.codfw.wmnet to elasticsearch 1.7.5 |
[production] |
| 23:05 |
<ebernhardson> |
upgrade elastic2006.codfw.wmnet to elasticsearch 1.7.5 |
[production] |
| 22:56 |
<ori@tin> |
Synchronized wmf-config/StartProfiler.php: I6d3ab949a6a: Apply xhgui role on tungsten (duration: 00m 56s) |
[production] |
| 22:28 |
<ostriches> |
phabricator: stopped phd daemons for the time being so replication can catch up and number of connections stop yelling ad people |
[production] |
| 22:25 |
<ebernhardson> |
upgrade elastic2006.codfw.wmnet to elasticsearch 1.7.5 |
[production] |
| 22:14 |
<jynus> |
setting innodb_flush_log_at_trx_commit=0 on phabricator db slave |
[production] |
| 21:37 |
<ebernhardson> |
upgrade elastic2005.codfw.wmnet to elastic 1.7.5 |
[production] |
| 21:37 |
<arlolra> |
updated Parsoid to version d809ad7a |
[production] |
| 21:24 |
<elukey@tin> |
Synchronized wmf-config/filebackend-production.php: Add mc1001 from the lock managers after maintenance (duration: 00m 54s) |
[production] |
| 20:56 |
<elukey> |
Added mc1001.eqiad back into memcached/redis pools after maintenance |
[production] |
| 20:21 |
<gehel> |
elastic2004.codfw.wmnet: upgrading to 1.7.5, shipping logs to logstash (T122697, T109101) |
[production] |
| 20:04 |
<ori> |
Restarted statsdlb on graphite1001 with xxh64() improvement |
[production] |
| 19:45 |
<mutante> |
new planets, Albanian and Bulgarian: https://sq.planet.wikimedia.org/ | https://bg.planet.wikimedia.org/ |
[production] |
| 19:25 |
<elukey> |
removed mc1001.equiad from the redis/memcached pools for maintenance |
[production] |
| 19:22 |
<gehel> |
elastic2003.codfw.wmnet: upgrading to 1.7.5, shipping logs to logstash (T122697, T109101) |
[production] |
| 19:13 |
<ori@tin> |
Synchronized wmf-config/filebackend-production.php: I0ad3e23719c: Follow-up for I8a87a679ed: unlist rdb1 server (duration: 00m 44s) |
[production] |
| 19:07 |
<elukey@tin> |
Synchronized wmf-config/filebackend-production.php: Remove mc1001 from the lock managers for maintenance (duration: 00m 41s) |
[production] |
| 18:40 |
<mattflaschen@tin> |
Synchronized dblists/nonflow.dblist: Re-enable Flow on ptwikibooks. Accidentally disabled earlier. (duration: 00m 47s) |
[production] |
| 18:05 |
<krinkle@tin> |
Synchronized w/static.php: (no message) (duration: 01m 01s) |
[production] |
| 18:01 |
<gehel> |
elastic2002.codfw.wmnet: upgrading to 1.7.5, shipping logs to logstash (T122697, T109101) |
[production] |
| 16:43 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Set transwiki import sources for hi.wikiquote [[gerrit:272992]] (duration: 00m 41s) |
[production] |
| 16:38 |
<thcipriani@tin> |
Synchronized wmf-config/throttle.php: SWAT: Maintenance on throttle.php [[gerrit:273580]] (duration: 00m 41s) |
[production] |
| 16:32 |
<thcipriani@tin> |
Synchronized php-1.27.0-wmf.14/extensions/Flow: SWAT: Fix board move DB issue using new hook TitleMoveStarting [[gerrit:273536]] (duration: 01m 00s) |
[production] |
| 16:28 |
<thcipriani@tin> |
Synchronized php-1.27.0-wmf.14/extensions/Wikidata: SWAT: Update Wikibase: Fix over-encoding of expanded URLs [[gerrit:273917]] (duration: 02m 04s) |
[production] |
| 16:23 |
<thcipriani@tin> |
Synchronized php-1.27.0-wmf.14/includes/MovePage.php: SWAT: Add TitleMoveStarting, mirroring TitleMoveCompleting [[gerrit:273535]] (duration: 00m 41s) |
[production] |
| 16:09 |
<thcipriani@tin> |
Synchronized wmf-config/CommonSettings.php: SWAT: Temporarily disable thank-you-edit notifications [[gerrit:273836]] (duration: 00m 41s) |
[production] |
| 16:06 |
<thcipriani@tin> |
Synchronized wmf-config/Wikibase.php: SWAT: Update Wikidata property blacklist [[gerrit:271971]] (duration: 01m 01s) |
[production] |
| 16:02 |
<_joe_> |
stopping hhvm on mw1050 (depooled) for testing bug T128380 |
[production] |
| 15:50 |
<urandom> |
Rolling restart of restbase production complete : T103124 |
[production] |
| 15:43 |
<urandom> |
Perform rolling restart of restbase in production cluster : T103124 |
[production] |
| 15:42 |
<urandom> |
Rolling restart of restbase staging complete : T103124 |
[production] |
| 15:33 |
<urandom> |
Perform rolling restart of restbase in staging cluster : T103124 |
[production] |
| 15:32 |
<urandom> |
forcing puppet run in restbase clustter ((noop) config deploy) : T103124 |
[production] |
| 15:30 |
<urandom> |
forcing puppet run in restbase staging ((noop) config deploy) : T103124 |
[production] |
| 14:17 |
<bblack> |
upgrading nginx on cp* to 1.9.4-1+wmf2 for T126616 |
[production] |
| 14:16 |
<gehel> |
elastic2001.codfw.wmnet: upgrading to 1.7.5, shipping logs to logstash (T122697, T109101) |
[production] |
| 14:11 |
<gehel> |
re-enabling icinga notifications for elasticsearch cluster on codfw |
[production] |
| 14:00 |
<gehel> |
icinga: adding authorization for Gehel |
[production] |
| 13:54 |
<bblack> |
puppet back to normal on caches |
[production] |
| 13:35 |
<bblack> |
disabling puppet on cp* for traffic-pool work |
[production] |
| 12:50 |
<godog> |
gmetad on uranium restarted following https://gerrit.wikimedia.org/r/273881 it will converge asap |
[production] |
| 12:16 |
<gehel> |
elastic2001.codfw.wmnet: upgrading to 1.7.5, shipping logs to logstash (T122697, T109101) |
[production] |
| 11:57 |
<jynus> |
reimaging dbproxy1005 with jessie before its put into production (potential alerts due to new role) |
[production] |
| 10:46 |
<ema> |
hhvm restarted on mw1119 |
[production] |
| 10:25 |
<jynus> |
powercycling mw1140, almost 100% unresponsive, OOM probable cause |
[production] |
| 09:58 |
<godog> |
bootstrap restbase1009-b T95253 |
[production] |