|
2016-09-20
§
|
| 10:44 |
<moritzm> |
reimaging app servers mw1240-mw1242 and API servers mw1194/mw1195 to jessie |
[production] |
| 10:43 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: (no message) (duration: 00m 48s) |
[production] |
| 10:36 |
<moritzm> |
upgrading ganeti2004 to Linux 4.4 |
[production] |
| 10:28 |
<akosiaris> |
force mw2232 to use palladium for report handler testing |
[production] |
| 10:25 |
<jynus> |
deploying schema change on s1 hosts T139090 |
[production] |
| 10:15 |
<moritzm> |
upgrading ganeti2003 to Linux 4.4 |
[production] |
| 10:12 |
<mobrovac> |
change-prop deploying e1ef51e |
[production] |
| 09:57 |
<moritzm> |
upgrading ganeti2002 to Linux 4.4 |
[production] |
| 09:13 |
<moritzm> |
reimaging API servers mw1192/mw1193 to jessie |
[production] |
| 08:55 |
<moritzm> |
reimaging mw1243-mw1245 to jessie |
[production] |
| 07:40 |
<elukey> |
restart cassandra on aqs100[456] for T130861 - only aqs1004 is taking live traffic |
[analytics] |
| 07:36 |
<elukey> |
restart cassandra on aqs100[456] for T130861 - only aqs1004 is taking live traffic |
[production] |
| 02:47 |
<eileen> |
CiviCRM update from c9157ba6aac3ad3c2fd61c7bb0475851fb8ec421 to de1df9ec4fe58487eeb61cc69160f228e009f2cf |
[production] |
| 02:40 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Sep 20 02:40:17 UTC 2016 (duration 6m 52s) |
[production] |
| 02:33 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.18) (duration: 10m 35s) |
[production] |
| 00:43 |
<mutante> |
wtp2019 - down again, powercycled, probably damaged RAM |
[production] |
|
2016-09-19
§
|
| 23:22 |
<Pchelolo> |
restart restbase in staging |
[production] |
| 23:01 |
<awight> |
update orphan rectifier config for T145848 |
[production] |
| 22:58 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: Restore testwiki to 1.28.0-wmf.18 |
[production] |
| 22:53 |
<dapatrick> |
Deployed patch for T144573 to wmf18 and wmf19 |
[production] |
| 22:48 |
<thcipriani@tin> |
Finished scap: testwiki to php-1.28.0-wmf.19 and rebuild l10n cache (duration: 52m 03s) |
[production] |
| 22:31 |
<bblack> |
cache_upload: pooling cp1099 (storage experiment - T145661) |
[production] |
| 22:02 |
<awight> |
update paymentswiki from 4cd18776ca67fe0db04b4173cb47f39dda59f43b to d572ee9e3ef0a97c044f02e9866469a8f3fa5858 |
[production] |
| 22:01 |
<legoktm> |
shutdown integration-puppetmaster |
[releng] |
| 22:00 |
<bblack> |
cp1099: depooling varnish backends for storage size experimentation |
[production] |
| 21:56 |
<thcipriani@tin> |
Started scap: testwiki to php-1.28.0-wmf.19 and rebuild l10n cache |
[production] |
| 21:41 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: Revert group0 wikis to 1.28.0-wmf.19 |
[production] |
| 21:30 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group0 wikis to 1.28.0-wmf.19 |
[production] |
| 21:29 |
<yuvipanda> |
regenerated client certs only on integration-puppetmaster01, seems ok now |
[releng] |
| 20:46 |
<yuvipanda> |
re-enable puppet everywhere |
[releng] |
| 20:43 |
<yuvipanda> |
enable puppet and run on integration-slave-trusty-1003.eqiad.wmflabs |
[releng] |
| 20:42 |
<yuvipanda> |
accidentally deleted /var/lib/puppet/ssl on integration-puppetmaster01 as well, causing it to lose keys. Reprovision by pointing to labs puppetmaster |
[releng] |
| 20:40 |
<hoo> |
Removed today's Wikidata json dumps: All shards succeeded, but final dump composition apparently failed. |
[production] |
| 20:34 |
<yuvipanda> |
rm -rf /var/lib/puppet/ssl on all integration nodes |
[releng] |
| 20:34 |
<yuvipanda> |
copied /etc/puppet/puppet.conf from integration-trusty-slave-1001 to all integration |
[releng] |
| 20:25 |
<yuvipanda> |
delete /etc/puppet/puppet.conf.d/10-self.conf and /var/lib/puppet/ssl on integration-slave-trusty-1001 |
[releng] |
| 20:20 |
<yuvipanda> |
re-enabled puppet on integration-slave-trusty-1001 |
[releng] |
| 20:14 |
<chasemp> |
reboot labstore1004 |
[production] |
| 20:08 |
<yuvipanda> |
reset puppetmaster of integration-puppetmaster01 to be labs puppetmaster |
[releng] |
| 20:08 |
<Pchelolo> |
restbase deploy 4829630f staging |
[production] |
| 20:03 |
<yuvipanda> |
disable puppet across integration project, moving puppetmasters |
[releng] |
| 19:49 |
<legoktm> |
creating T144951 enabled role::puppetmaster::standalone role on integration-puppetmaster01 |
[releng] |
| 19:33 |
<legoktm> |
creating T144951 integration-puppetmaster01 instance using m1.small and debian jessie |
[releng] |
| 18:49 |
<thcipriani@tin> |
Synchronized wmf-config/throttle.php: SWAT: [[gerrit:311417|Throttle for RCL (T145838)]] (duration: 00m 47s) |
[production] |
| 18:43 |
<thcipriani@tin> |
Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:311229|ORES default threshold to high for wikidatawiki (T144784)]] (duration: 00m 47s) |
[production] |
| 18:32 |
<jynus> |
emergency/unscheduled restart of mariadb @ labsdb1003 - close to OOM, unusable |
[production] |
| 18:14 |
<thcipriani> |
ran on terbium: mwscript extensions/ShortUrl/populateShortUrlTable.php --wiki=bdwikimedia |
[production] |
| 17:47 |
<awight> |
reenable banner history queue consumer |
[production] |
| 17:46 |
<awight> |
update civicrm from 1df25962834885121bd7a9ba856c0c69c2b9cfda to c9157ba6aac3ad3c2fd61c7bb0475851fb8ec421 |
[production] |
| 17:39 |
<ejegg> |
updated payments-wiki from 392d67520d14998d61823755b22df50ab45afb35 to 4cd18776ca67fe0db04b4173cb47f39dda59f43b |
[production] |