|
2014-10-09
§
|
| 17:55 |
<Coren> |
done reimaging of mw1028. Now hhvm_appserver |
[production] |
| 16:58 |
<_joe_> |
gracefully restarted again api apaches to recover 500s |
[production] |
| 16:43 |
<godog> |
re-enable puppet on ms-fe/ms-be in eqiad |
[production] |
| 16:39 |
<godog> |
re-enable puppet on ms-fe/ms-be in codfw |
[production] |
| 16:23 |
<oblivian> |
gracefulled all apaches |
[production] |
| 15:34 |
<hashar> |
restarted Zuul |
[production] |
| 15:31 |
<Coren> |
begin reimaging of mw1028 |
[production] |
| 15:31 |
<Coren> |
done reimaging of mw1029. Now hhvm_appserver |
[production] |
| 15:28 |
<manybubbles> |
Synchronized php-1.25wmf2/extensions/VisualEditor/: SWAT deploy VE cherry-pick (duration: 00m 06s) |
[production] |
| 15:26 |
<andrewbogott> |
upgraded wikitech-static to 1.25wmf2 |
[production] |
| 14:02 |
<akosiaris> |
updated pybal on palladium for citoid |
[production] |
| 13:54 |
<Coren> |
begin reimaging of mw1029 |
[production] |
| 12:02 |
<reedy> |
Purged l10n cache for 1.24wmf22 |
[production] |
| 11:59 |
<springle> |
converted some librenms tables to innodb on db1001 m1-master. should be a no-op |
[production] |
| 11:57 |
<springle> |
xtrabackup db1016 to db2010 |
[production] |
| 11:39 |
<manybubbles> |
starting upgrade of elastic1009 |
[production] |
| 11:12 |
<_joe_> |
reenabled puppet on mw* |
[production] |
| 11:11 |
<godog> |
disabled puppet in ms-fe/ms-be in eqiad/codfw to merge container-sync changes |
[production] |
| 10:35 |
<_joe_> |
disabling puppet on most mw* hosts while testing apache changes |
[production] |
| 08:17 |
<_joe_> |
repooling mw102[3-5],mw1053 in the hhvm pool |
[production] |
| 07:15 |
<_joe_> |
reimaging mw102[3-5] to hhvm |
[production] |
| 07:02 |
<_joe_> |
reinstalling mw1053 |
[production] |
| 03:33 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Oct 9 03:33:47 UTC 2014 (duration 33m 46s) |
[production] |
| 02:30 |
<LocalisationUpdate> |
completed (1.25wmf2) at 2014-10-09 02:30:03+00:00 |
[production] |
| 02:18 |
<LocalisationUpdate> |
completed (1.25wmf1) at 2014-10-09 02:17:53+00:00 |
[production] |
|
2014-10-08
§
|
| 23:31 |
<mutante> |
importing xml dump to cawikimedia |
[production] |
| 23:28 |
<andrewbogott> |
running sync-common on virt1000 |
[production] |
| 23:27 |
<demon> |
Synchronized php-1.25wmf2/extensions/OpenStackManager: (no message) (duration: 00m 06s) |
[production] |
| 23:23 |
<demon> |
Synchronized php-1.25wmf2/extensions/Flow: (no message) (duration: 00m 05s) |
[production] |
| 23:14 |
<demon> |
Synchronized php-1.25wmf2/extensions/CommonsMetadata: (no message) (duration: 00m 06s) |
[production] |
| 22:40 |
<mutante> |
virt0 - deleted salt key, revoked puppet cert, removed from site.pp |
[production] |
| 22:39 |
<Reedy> |
Removed openjdk-6-* from logstash100[1-3] |
[production] |
| 22:07 |
<subbu> |
updated OCG to version def24eca |
[production] |
| 22:07 |
<mutante> |
tin - deleted empty pmtpa dsh group files |
[production] |
| 22:02 |
<mutante> |
tin - there are dozens of dsh groups that have been removed from repo long time ago but never got purged, but it isn't easy to tell what might still be used, so deleting all and letting puppet recreate might be risky? |
[production] |
| 20:48 |
<legoktm> |
currently running /home/legoktm/fixBug71749.php on terbium |
[production] |
| 19:49 |
<Reedy> |
logstash upgraded to 1.4.2-1 on logstash100[1-3] |
[production] |
| 19:46 |
<Reedy> |
Created flow tables on officewiki |
[production] |
| 17:04 |
<_joe_> |
load testing done |
[production] |
| 16:44 |
<_joe_> |
doing some load testing on the hhvm servers |
[production] |
| 16:09 |
<Reedy> |
elasticsearch upgraded on logstash1001 to 1.3.4 |
[production] |
| 16:08 |
<Reedy> |
elasticsearch upgraded on logstash100[23] to 1.3.4 |
[production] |
| 16:07 |
<Reedy> |
elasticsearch upgraded on logstash[ |
[production] |
| 15:08 |
<anomie> |
Synchronized php-1.25wmf2/extensions/VisualEditor/lib/ve/src/ce/ve.ce.Surface.js: SWAT: Revert "ve.ce.Surface: Magic workaround for broken Firefox cursoring" [[gerrit:164593]] (duration: 00m 09s) |
[production] |
| 15:04 |
<manybubbles> |
upgrading elastic1002 now |
[production] |
| 14:53 |
<_joe_> |
repooling mw1163 |
[production] |
| 14:20 |
<hasharBusy> |
disabled puppet on gallium to make sure a zuul config change stick in. {{gerrit|165481}} |
[production] |
| 14:19 |
<manybubbles> |
fixed missing elasticsearch extension jar file and brought elastic1001 back up. git fat betrayed us. |
[production] |
| 14:14 |
<hasharBusy> |
hard restarting zuul |
[production] |
| 14:03 |
<manybubbles> |
upgrading elastic1001 uncovered a bug in our highlighter that I have yet to diagnose. I removed that server from the rotation so we'll continue to use the old version. |
[production] |