|
2015-02-06
§
|
| 16:40 |
<godog> |
cancel downtime on graphite1001, enable downtime on tungsten pending full decomission |
[production] |
| 16:05 |
<godog> |
bounce ocg on ocg1001 and stop additional ocg instance running |
[production] |
| 15:50 |
<bblack> |
depool -> repool cp1064 varnish-frontend, reduced cache size to 16G, re-enabled compact_memory |
[production] |
| 15:50 |
<godog> |
restart ocg on ocg1003 to pick up statsd dns changes |
[production] |
| 15:48 |
<godog> |
restart ocg on ocg1002 to pick up statsd dns changes |
[production] |
| 14:33 |
<bblack> |
starting up a fresh round of SSL testing on eqiad upload pooling (cp1064) |
[production] |
| 14:12 |
<godog> |
bounce diamond on lvs2004/lvs2005 |
[production] |
| 13:55 |
<cmjohnson1> |
upgrading boron to trusty |
[production] |
| 10:51 |
<godog> |
reimage ms-be2014 |
[production] |
| 07:50 |
<_joe_> |
restarting the parsoid cluster, one node at a time, some processes are stuck. |
[production] |
| 02:04 |
<LocalisationUpdate> |
failed: git pull of extensions failed |
[production] |
| 01:01 |
<ori> |
restarting xenon on fluorine |
[production] |
| 00:14 |
<krenair> |
Synchronized php-1.25wmf15/includes/CategoryViewer.php: https://gerrit.wikimedia.org/r/#/c/188944/1 (duration: 00m 06s) |
[production] |
| 00:13 |
<krenair> |
Synchronized php-1.25wmf16/includes/CategoryViewer.php: https://gerrit.wikimedia.org/r/#/c/188945/1 (duration: 00m 06s) |
[production] |
|
2015-02-05
§
|
| 23:53 |
<bd808> |
Updated Wikimania Scholarships to 0852585 (re-enable language selection) + local hack in trebuchet repo to remove incomplete translations |
[production] |
| 22:53 |
<reedy> |
Synchronized wmf-config/InitialiseSettings.php: Enable Parsoid on wikitech (duration: 00m 05s) |
[production] |
| 21:08 |
<reedy> |
Purged l10n cache for 1.25wmf13 |
[production] |
| 21:03 |
<reedy> |
Synchronized php-1.25wmf16/extensions/CheckUser/: (no message) (duration: 00m 07s) |
[production] |
| 21:00 |
<reedy> |
Synchronized php-1.25wmf16: (no message) (duration: 00m 52s) |
[production] |
| 20:59 |
<bblack> |
cp1064 upload b ackend re-enabled in cache.pp; if upload-related 503s ensue later today and I'm not around, feel free to re-disable it |
[production] |
| 20:57 |
<mutante> |
radon - reinstalling, scheduled downtime |
[production] |
| 20:42 |
<Reedy> |
mw1092 giving file has vanished: "/wmf-config/.InitialiseSettings.php.KSg3AF" (in common) |
[production] |
| 20:42 |
<reedy> |
Synchronized wmf-config/: GlobalUserPage and I33a855cecfbe25003fe9e4f5e2fab2f928c79da4 (duration: 00m 07s) |
[production] |
| 20:41 |
<reedy> |
Synchronized wmf-config/: GlobalUserPage and I33a855cecfbe25003fe9e4f5e2fab2f928c79da4 (duration: 00m 05s) |
[production] |
| 20:35 |
<reedy> |
Synchronized wmf-config/: GlobalUserPage and I33a855cecfbe25003fe9e4f5e2fab2f928c79da4 (duration: 00m 08s) |
[production] |
| 20:34 |
<reedy> |
Synchronized php-1.25wmf16/includes/EditPage.php: Id376f9e75c43c5bd0fa910b04d066e6aa37c73d1 (duration: 00m 07s) |
[production] |
| 20:30 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.25wmf16 |
[production] |
| 20:29 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.25wmf15 |
[production] |
| 20:21 |
<reedy> |
Finished scap: testwiki to 1.25wmf16 (duration: 38m 37s) |
[production] |
| 20:08 |
<_joe_> |
re-exported nfs exports on dataset1001, remounted /mnt/data on snapshot1001 |
[production] |
| 19:42 |
<reedy> |
Started scap: testwiki to 1.25wmf16 |
[production] |
| 19:41 |
<reedy> |
scap aborted: testwiki to 1.25wmf16 (duration: 03m 27s) |
[production] |
| 19:37 |
<reedy> |
Started scap: testwiki to 1.25wmf16 |
[production] |
| 19:10 |
<legoktm> |
clearing bad sidebar memcache entries on commonswiki |
[production] |
| 18:08 |
<_joe_> |
restarting nutcracker on jobrunners |
[production] |
| 18:06 |
<_joe_> |
restarting nutcracker on api appservers |
[production] |
| 18:01 |
<ori> |
restarting nutcracker on all appservers |
[production] |
| 17:54 |
<ori> |
Synchronized wmf-config/InitialiseSettings.php: I4f28205e6: Set $wmgUseMonologLogger to false (duration: 00m 06s) |
[production] |
| 17:47 |
<ori> |
Synchronized wmf-config/logging.php: Live hack: disable Logstash logging on suspicion that it is acting up (duration: 00m 05s) |
[production] |
| 17:34 |
<paravoid> |
restarting HHVM on all appservers/API appservers in 10%/6s batches |
[production] |
| 17:26 |
<bblack> |
repooled cp1063 frontend-only |
[production] |
| 16:21 |
<godog> |
bounce jmxtrans on analytics1018, analytics1021 and analytics1022 |
[production] |
| 16:15 |
<godog> |
bounce jmxtrans on analytics1012 |
[production] |
| 16:03 |
<godog> |
re-enabled puppet on graphite1001, bounce uwsgi |
[production] |
| 14:34 |
<godog> |
upload txstatsd 1.0.0-3 to trusty-wikimedia |
[production] |
| 12:42 |
<paravoid> |
cp*/amssq*: salt rm /etc/logrotate.d/varnishkafka-frontend-stats to fix cronspam |
[production] |
| 12:30 |
<hashar> |
Upgrading Jenkins and restarting it |
[production] |
| 06:43 |
<springle> |
upgrade silver to mariadb 10 |
[production] |
| 04:55 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Feb 5 04:54:00 UTC 2015 (duration 53m 59s) |
[production] |
| 02:37 |
<ori> |
Synchronized wmf-config/CommonSettings.php: Ia59e654e8: Set a statsd-compatible $wgStatsFormatString (duration: 00m 07s) |
[production] |