2015-10-23
§
|
23:31 |
<Krinkle> |
mwscript deleteEqualMessages.php --wiki hewikisource |
[production] |
23:31 |
<Krinkle> |
mwscript deleteEqualMessages.php --wiki ukwiki |
[production] |
23:09 |
<hoo> |
Started a local rename for Stefan4 to Stefan2 on commons, per request (and attributed to) DerHexer |
[production] |
22:23 |
<robh> |
powercycled nobelium |
[production] |
21:24 |
<mutante> |
nobelium - powercycled, no console output |
[production] |
20:04 |
<jynus> |
rename user job had created lag on almost all enwiki dbs, things should be better now |
[production] |
19:08 |
<urandom> |
starting nodetool cleanup on restbase-test2001-b |
[production] |
19:07 |
<urandom> |
starting nodetool cleanup on restbase-test2001-a |
[production] |
18:16 |
<godog> |
bounce grafana-server on krypton |
[production] |
17:58 |
<YuviPanda> |
moved mounts around on nobelium, mounted bigger disk on /var/lib/elasticsearch |
[production] |
17:57 |
<godog> |
delete wikimedia-grid grafana dashboard (saved a copy first) heavy graphite queries |
[production] |
17:53 |
<YuviPanda> |
stopping elasticsearch on nobelium for https://phabricator.wikimedia.org/T114856 |
[production] |
17:13 |
<akosiaris> |
enable_notifications=0 in neon's icinga for a few mins while the storm dies down |
[production] |
16:47 |
<godog> |
stop puppet on graphite1001 and graphite-index cron, suspected root cause |
[production] |
16:26 |
<godog> |
uwsgi timing out while serving requests, bounce also carbon daemons |
[production] |
16:20 |
<godog> |
bounce uwsgi for graphite-web on graphite1001 |
[production] |
16:14 |
<urandom> |
starting rebuild of restbase-test2002-b |
[production] |
15:42 |
<jynus> |
copied wmf-mariadb10 (10.0.16-2) .deb from trusty to jessie on apt.wikimedia.org |
[production] |
15:42 |
<urandom> |
bouncing restbase-test2002-a Just To See |
[production] |
15:06 |
<urandom> |
running nodetool cleanup on restbase-test2003 |
[production] |
14:47 |
<godog> |
remove outdated cassandra metrics from graphite2001 |
[production] |
14:27 |
<godog> |
remove restbase-test2001 restbase-test2002 cassandra metrics |
[production] |
14:12 |
<mobrovac> |
restbase rolling-restarting after applying aab840f |
[production] |
13:55 |
<jynus> |
Rebooting and installing jessie on db2055-db2059 |
[production] |
13:45 |
<godog> |
remove 98percentile 999percentile meanRate stddev cassandra metrics after https://gerrit.wikimedia.org/r/#/c/248313/ |
[production] |
13:33 |
<godog> |
remove 15MinuteRate cassandra metrics after https://gerrit.wikimedia.org/r/#/c/248313/ |
[production] |
11:51 |
<godog> |
roll-restart cassandra-metrics-collector on restbase cluster after https://gerrit.wikimedia.org/r/248313 |
[production] |
11:14 |
<jynus> |
After days of no references on the logs to the renamed user_daily_contribs, deleting delete_user_daily_contribs table on all wikis |
[production] |
10:27 |
<jynus> |
retrying schema change on EventLogging (db1046) after failure |
[production] |
10:21 |
<jynus> |
End of online schema change to geo_tags; all wikis on dblist have been updated |
[production] |
09:34 |
<godog> |
reimage restbase-test2002.codfw.wmnet |
[production] |
09:23 |
<jynus> |
restarting schema change on geo_tags for all wikis |
[production] |
09:17 |
<mobrovac> |
restbase rolling-restart after config changes |
[production] |
06:55 |
<ori@tin> |
Synchronized php-1.27.0-wmf.3/extensions/AbuseFilter/AbuseFilterTokenizer.php: I65d4c6064: Track tokenizer cache hits / misses (duration: 00m 18s) |
[production] |
06:55 |
<ori@tin> |
Synchronized php-1.27.0-wmf.2/extensions/AbuseFilter/AbuseFilterTokenizer.php: I65d4c6064: Track tokenizer cache hits / misses (duration: 00m 17s) |
[production] |
05:53 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Fri Oct 23 05:53:43 UTC 2015 (duration 53m 42s) |
[production] |
03:15 |
<aaron@tin> |
Synchronized php-1.27.0-wmf.3/includes/profiler/TransactionProfiler.php: 5ef4a91480ea (duration: 00m 18s) |
[production] |
02:41 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.27.0-wmf.3) at 2015-10-23 02:41:25+00:00 |
[production] |
02:36 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.3/cache/l10n: l10nupdate for 1.27.0-wmf.3 (duration: 08m 04s) |
[production] |
00:10 |
<krenair@tin> |
Synchronized w/static/images/project-logos/anwiki.png: https://gerrit.wikimedia.org/r/#/c/247253/ (duration: 00m 16s) |
[production] |