|
2016-09-20
§
|
| 17:01 |
<elukey> |
adding aqs1006 to live traffic - aqs.svc.eqiad.wmnet - T144497 |
[production] |
| 16:58 |
<elukey> |
adding aqs1005 to live traffic - aqs.svc.eqiad.wmnet - T144497 |
[production] |
| 16:48 |
<gehel> |
increase recovery bandwidth on elasticsearch eqiad to match codfw - T145404 |
[production] |
| 16:32 |
<elukey> |
restarting cassandra on aqs100[56] (started the work earlier on today, stopped due to T146130) |
[production] |
| 14:31 |
<dcausse> |
restarting relforge100[12].eqiad.wmnet servers for kernel upgrade and java settings change |
[production] |
| 14:23 |
<moritzm> |
installing tomcat security updates on Ubuntu servers |
[production] |
| 13:41 |
<jynus> |
disabling puppet on labtestweb2001 |
[production] |
| 13:40 |
<ottomata> |
merged --until flag change in check_graphite script (this could affect all graphite based alerts) |
[production] |
| 13:37 |
<zeljkof> |
executed script: mwscript maintenance/updateArticleCount.php --wiki=wikidatawiki --update |
[production] |
| 13:34 |
<zeljkof> |
EU SWAT finished |
[production] |
| 13:28 |
<gehel> |
restarting for elasticsearch and kernel upgrade - eqiad cluster - T145404 / T146123 |
[production] |
| 13:05 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:308430|Change $wgArticleCountMethod in Wikidata from default (link) to any (T144687)]] (duration: 00m 47s) |
[production] |
| 11:41 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: (no message) (duration: 00m 46s) |
[production] |
| 11:38 |
<moritzm> |
upgrading ganeti2001 to Linux 4.4 (ganeti2006 has been promoted to new master node) |
[production] |
| 11:05 |
<moritzm> |
upgrading ganeti2006 to Linux 4.4 |
[production] |
| 10:49 |
<moritzm> |
upgrading ganeti2005 to Linux 4.4 |
[production] |
| 10:44 |
<moritzm> |
reimaging app servers mw1240-mw1242 and API servers mw1194/mw1195 to jessie |
[production] |
| 10:43 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: (no message) (duration: 00m 48s) |
[production] |
| 10:36 |
<moritzm> |
upgrading ganeti2004 to Linux 4.4 |
[production] |
| 10:28 |
<akosiaris> |
force mw2232 to use palladium for report handler testing |
[production] |
| 10:25 |
<jynus> |
deploying schema change on s1 hosts T139090 |
[production] |
| 10:15 |
<moritzm> |
upgrading ganeti2003 to Linux 4.4 |
[production] |
| 10:12 |
<mobrovac> |
change-prop deploying e1ef51e |
[production] |
| 09:57 |
<moritzm> |
upgrading ganeti2002 to Linux 4.4 |
[production] |
| 09:13 |
<moritzm> |
reimaging API servers mw1192/mw1193 to jessie |
[production] |
| 08:55 |
<moritzm> |
reimaging mw1243-mw1245 to jessie |
[production] |
| 07:36 |
<elukey> |
restart cassandra on aqs100[456] for T130861 - only aqs1004 is taking live traffic |
[production] |
| 02:47 |
<eileen> |
CiviCRM update from c9157ba6aac3ad3c2fd61c7bb0475851fb8ec421 to de1df9ec4fe58487eeb61cc69160f228e009f2cf |
[production] |
| 02:40 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Sep 20 02:40:17 UTC 2016 (duration 6m 52s) |
[production] |
| 02:33 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.18) (duration: 10m 35s) |
[production] |
| 00:43 |
<mutante> |
wtp2019 - down again, powercycled, probably damaged RAM |
[production] |
|
2016-09-19
§
|
| 23:22 |
<Pchelolo> |
restart restbase in staging |
[production] |
| 23:01 |
<awight> |
update orphan rectifier config for T145848 |
[production] |
| 22:58 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: Restore testwiki to 1.28.0-wmf.18 |
[production] |
| 22:53 |
<dapatrick> |
Deployed patch for T144573 to wmf18 and wmf19 |
[production] |
| 22:48 |
<thcipriani@tin> |
Finished scap: testwiki to php-1.28.0-wmf.19 and rebuild l10n cache (duration: 52m 03s) |
[production] |
| 22:31 |
<bblack> |
cache_upload: pooling cp1099 (storage experiment - T145661) |
[production] |
| 22:02 |
<awight> |
update paymentswiki from 4cd18776ca67fe0db04b4173cb47f39dda59f43b to d572ee9e3ef0a97c044f02e9866469a8f3fa5858 |
[production] |
| 22:00 |
<bblack> |
cp1099: depooling varnish backends for storage size experimentation |
[production] |
| 21:56 |
<thcipriani@tin> |
Started scap: testwiki to php-1.28.0-wmf.19 and rebuild l10n cache |
[production] |
| 21:41 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: Revert group0 wikis to 1.28.0-wmf.19 |
[production] |
| 21:30 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group0 wikis to 1.28.0-wmf.19 |
[production] |
| 20:40 |
<hoo> |
Removed today's Wikidata json dumps: All shards succeeded, but final dump composition apparently failed. |
[production] |
| 20:14 |
<chasemp> |
reboot labstore1004 |
[production] |
| 20:08 |
<Pchelolo> |
restbase deploy 4829630f staging |
[production] |
| 18:49 |
<thcipriani@tin> |
Synchronized wmf-config/throttle.php: SWAT: [[gerrit:311417|Throttle for RCL (T145838)]] (duration: 00m 47s) |
[production] |
| 18:43 |
<thcipriani@tin> |
Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:311229|ORES default threshold to high for wikidatawiki (T144784)]] (duration: 00m 47s) |
[production] |
| 18:32 |
<jynus> |
emergency/unscheduled restart of mariadb @ labsdb1003 - close to OOM, unusable |
[production] |
| 18:14 |
<thcipriani> |
ran on terbium: mwscript extensions/ShortUrl/populateShortUrlTable.php --wiki=bdwikimedia |
[production] |
| 17:47 |
<awight> |
reenable banner history queue consumer |
[production] |