|
2016-04-26
§
|
| 12:09 |
<elukey> |
mw1145 powercycled (root login timeout on the console) |
[production] |
| 11:43 |
<gehel> |
restarting elasticsearch server elastic2009.codfw.wmnet - activating unicast (T110236) |
[production] |
| 10:31 |
<jynus> |
restarting mw1143 as is no longer accessible |
[production] |
| 09:51 |
<jynus> |
restarting db2068 for upgrade before returning from maintenance |
[production] |
| 09:50 |
<gehel> |
starting reinstall of wdqs1001 (T133566) |
[production] |
| 09:47 |
<gehel> |
restarting elasticsearch server elastic2008.codfw.wmnet - activating unicast (T110236) |
[production] |
| 09:41 |
<moritzm> |
uploaded apache 2.4.10-10+deb8u4+wmf1 to carbon |
[production] |
| 09:12 |
<_joe_> |
restarted hhvm on mw1133, almost OOM |
[production] |
| 09:11 |
<hashar> |
CI is back up! |
[production] |
| 09:09 |
<mobrovac> |
citoid deployed 36c2bf02 |
[production] |
| 09:03 |
<jynus> |
restarting db1069 s1 instance- one query is "stuck", creating lag on labs |
[production] |
| 08:56 |
<gehel> |
depooling wdqs1001 from varnish for reinstall |
[production] |
| 08:51 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1052 (old s1-master) with low weight (duration: 00m 31s) |
[production] |
| 08:50 |
<YuviPanda> |
restarted nova-conductor & scheduler on labcontrol1001 for T133654 |
[production] |
| 08:45 |
<hashar> |
Most of CI is down / deadlocked due to wmflabs being unresponsive T133654 |
[production] |
| 08:44 |
<gehel> |
restarting elasticsearch server elastic2007.codfw.wmnet - activating unicast (T110236) |
[production] |
| 08:27 |
<jynus> |
stopping db2068 for cloning to db2047 |
[production] |
| 08:06 |
<hashar> |
CI jobs deadlocked due to castor being unavailable | https://phabricator.wikimedia.org/T133652 |
[production] |
| 06:52 |
<jynus> |
restarting db2047 for reimaging |
[production] |
| 02:46 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Apr 26 02:46:52 UTC 2016 (duration 9m 1s) |
[production] |
| 02:37 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.21) (duration: 19m 17s) |
[production] |
| 01:09 |
<paravoid> |
reinstalling bast4001 w/ jessie |
[production] |
|
2016-04-25
§
|
| 23:51 |
<dereckson@tin> |
Synchronized docroot/noc/conf/: noc.wikimedia.org update ([[Gerrit:285061]], [[Gerrit:285062]], [[Gerrit:281977]]) (duration: 00m 26s) |
[production] |
| 23:48 |
<dereckson@tin> |
Synchronized docroot/noc/createTxtFileSymlinks.sh: noc: PoolCounterSettings-eqiad.php → PoolCounterSettings.php ([[Gerrit:285062]], no-op) (duration: 00m 30s) |
[production] |
| 23:21 |
<Krenair> |
Deployed patch for T132653 |
[production] |
| 23:10 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/284831/ - Disable Echo survey on French wikis |
[production] |
| 22:30 |
<eileen> |
Updating civicrm |
[production] |
| 21:02 |
<gehel> |
adding cp10(46|47|59|60)\.eqiad\.wmnet to maps caching cluster (T109162) |
[production] |
| 21:02 |
<cscott> |
updated OCG to version 58a720508deb368abfb7652e6a8c7225f95402d2 |
[production] |
| 20:46 |
<csteipp> |
deployed updated patch for T98313 to include very headers |
[production] |
| 20:35 |
<gehel> |
restarting elasticsearch server elastic2006.codfw.wmnet - activating unicast (T110236) |
[production] |
| 20:07 |
<subbu> |
finished deploying parsoid version d5363193 |
[production] |
| 20:04 |
<subbu> |
synced new code and restarted parsoid on wtp1001 as a canary |
[production] |
| 20:01 |
<subbu> |
deploying parsoid version d5363193 |
[production] |
| 19:49 |
<gehel> |
restarting elasticsearch server elastic2005.codfw.wmnet - activating unicast (T110236) |
[production] |
| 19:39 |
<hashar> |
Nodepool now booting 2 trusty instances for CI needs. T133203 |
[production] |
| 19:27 |
<gehel> |
disabling puppet on cp1043/cp1044 while reconfiguring Maps caching servers |
[production] |
| 19:27 |
<gehel> |
start configuration of new maps caching servers (T109162) |
[production] |
| 19:08 |
<gehel> |
restarting elasticsearch server elastic2004.codfw.wmnet - activating unicast (T110236) |
[production] |
| 18:40 |
<legoktm@tin> |
Synchronized wmf-config: ⓛⓐⓑⓢ-ⓞⓝⓛⓨ, ⓝⓞ-ⓞⓟ (duration: 00m 40s) |
[production] |
| 18:38 |
<bblack> |
switching mobile hostnames to text IPs in DNS (10 min TTL) |
[production] |
| 18:21 |
<Pchelolo> |
finished update RESTBase to c1d5193 |
[production] |
| 18:09 |
<Pchelolo> |
update RESTBase to c1d5193 |
[production] |
| 17:55 |
<Pchelolo> |
update RESTBase to c1d5193, canary on restbase1007 |
[production] |
| 17:53 |
<jynus> |
wiping db1052 tendril's monitoring data due to 5.5 -> 5.6 incompatibility |
[production] |
| 17:48 |
<jynus> |
db1052 now in jessie |
[production] |
| 17:47 |
<Pchelolo> |
update RESTBase to c1d5193, canary on restbase1005 |
[production] |
| 17:26 |
<Pchelolo> |
update RESTBase to c1d5193, staging |
[production] |
| 17:22 |
<mutante> |
Ⓐ |
[production] |
| 17:20 |
<ori> |
😊 |
[production] |