|
2016-04-26
§
|
| 12:39 |
<hashar> |
starting cut of 1.27.0-wmf.22 branch ( poke ostriches ) |
[releng] |
| 12:27 |
<gehel> |
restarting elasticsearch server elastic2010.codfw.wmnet - activating unicast (T110236) |
[production] |
| 12:09 |
<elukey> |
mw1145 powercycled (root login timeout on the console) |
[production] |
| 11:43 |
<gehel> |
restarting elasticsearch server elastic2009.codfw.wmnet - activating unicast (T110236) |
[production] |
| 10:31 |
<jynus> |
restarting mw1143 as is no longer accessible |
[production] |
| 10:29 |
<hashar> |
restored integration/phpunit on CI slaves due to https://integration.wikimedia.org/ci/job/operations-mw-config-phpunit/ failling |
[releng] |
| 09:51 |
<jynus> |
restarting db2068 for upgrade before returning from maintenance |
[production] |
| 09:50 |
<gehel> |
starting reinstall of wdqs1001 (T133566) |
[production] |
| 09:47 |
<gehel> |
restarting elasticsearch server elastic2008.codfw.wmnet - activating unicast (T110236) |
[production] |
| 09:41 |
<moritzm> |
uploaded apache 2.4.10-10+deb8u4+wmf1 to carbon |
[production] |
| 09:12 |
<_joe_> |
restarted hhvm on mw1133, almost OOM |
[production] |
| 09:11 |
<hashar> |
CI is back up! |
[releng] |
| 09:11 |
<hashar> |
CI is back up! |
[production] |
| 09:09 |
<mobrovac> |
citoid deployed 36c2bf02 |
[production] |
| 09:03 |
<jynus> |
restarting db1069 s1 instance- one query is "stuck", creating lag on labs |
[production] |
| 08:56 |
<gehel> |
depooling wdqs1001 from varnish for reinstall |
[production] |
| 08:51 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1052 (old s1-master) with low weight (duration: 00m 31s) |
[production] |
| 08:50 |
<YuviPanda> |
restarted nova-conductor & scheduler on labcontrol1001 for T133654 |
[production] |
| 08:45 |
<hashar> |
Most of CI is down / deadlocked due to wmflabs being unresponsive T133654 |
[production] |
| 08:44 |
<gehel> |
restarting elasticsearch server elastic2007.codfw.wmnet - activating unicast (T110236) |
[production] |
| 08:27 |
<jynus> |
stopping db2068 for cloning to db2047 |
[production] |
| 08:20 |
<hashar> |
shutoff instance castor, does not seem to be able to start again :( | T133652 |
[releng] |
| 08:12 |
<hashar> |
hard rebooting castor instance | T133652 |
[releng] |
| 08:10 |
<hashar> |
soft rebooting castor instance | T133652 |
[releng] |
| 08:06 |
<hashar> |
CI jobs deadlocked due to castor being unavailable | https://phabricator.wikimedia.org/T133652 |
[releng] |
| 08:06 |
<hashar> |
CI jobs deadlocked due to castor being unavailable | https://phabricator.wikimedia.org/T133652 |
[production] |
| 06:52 |
<jynus> |
restarting db2047 for reimaging |
[production] |
| 02:46 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Apr 26 02:46:52 UTC 2016 (duration 9m 1s) |
[production] |
| 02:37 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.21) (duration: 19m 17s) |
[production] |
| 01:09 |
<paravoid> |
reinstalling bast4001 w/ jessie |
[production] |
| 00:46 |
<thcipriani> |
temporary keyholder fix in place in beta |
[releng] |
| 00:18 |
<thcipriani> |
beta-scap-eqiad failure due to bad keyholder-auth.d fingerprints |
[releng] |
|
2016-04-25
§
|
| 23:51 |
<dereckson@tin> |
Synchronized docroot/noc/conf/: noc.wikimedia.org update ([[Gerrit:285061]], [[Gerrit:285062]], [[Gerrit:281977]]) (duration: 00m 26s) |
[production] |
| 23:48 |
<dereckson@tin> |
Synchronized docroot/noc/createTxtFileSymlinks.sh: noc: PoolCounterSettings-eqiad.php → PoolCounterSettings.php ([[Gerrit:285062]], no-op) (duration: 00m 30s) |
[production] |
| 23:21 |
<Krenair> |
Deployed patch for T132653 |
[production] |
| 23:10 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/284831/ - Disable Echo survey on French wikis |
[production] |
| 22:30 |
<eileen> |
Updating civicrm |
[production] |
| 21:02 |
<gehel> |
adding cp10(46|47|59|60)\.eqiad\.wmnet to maps caching cluster (T109162) |
[production] |
| 21:02 |
<cscott> |
updated OCG to version 58a720508deb368abfb7652e6a8c7225f95402d2 |
[production] |
| 20:58 |
<cscott> |
updated OCG to version 58a720508deb368abfb7652e6a8c7225f95402d2 |
[releng] |
| 20:46 |
<csteipp> |
deployed updated patch for T98313 to include very headers |
[production] |
| 20:35 |
<gehel> |
restarting elasticsearch server elastic2006.codfw.wmnet - activating unicast (T110236) |
[production] |
| 20:07 |
<subbu> |
finished deploying parsoid version d5363193 |
[production] |
| 20:04 |
<subbu> |
synced new code and restarted parsoid on wtp1001 as a canary |
[production] |
| 20:01 |
<subbu> |
deploying parsoid version d5363193 |
[production] |
| 19:55 |
<nuria_> |
deployed new vitalsigns code to https://vital-signs.wmflabs.org |
[analytics] |
| 19:49 |
<gehel> |
restarting elasticsearch server elastic2005.codfw.wmnet - activating unicast (T110236) |
[production] |
| 19:46 |
<hashar> |
Nodepool now has a couple trusty instances intended to experiment with Zend 5.5 / HHVM migration . https://phabricator.wikimedia.org/T133203#2236625 |
[releng] |
| 19:39 |
<hashar> |
Nodepool now booting 2 trusty instances for CI needs. T133203 |
[production] |
| 19:27 |
<gehel> |
disabling puppet on cp1043/cp1044 while reconfiguring Maps caching servers |
[production] |