|
2015-07-07
§
|
| 11:31 |
<mobrovac> |
restbase restarted cassandra on rb1005 |
[production] |
| 11:26 |
<godog> |
restart cassandra on restbase1004, heap exhausted |
[production] |
| 10:49 |
<godog> |
restarted cassandra on restbase1005, mutations through the roof |
[production] |
| 08:27 |
<godog> |
set operations/puppet/cassandra git submodule repo as hidden |
[production] |
| 06:11 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue Jul 7 06:11:46 UTC 2015 (duration 11m 45s) |
[production] |
| 05:51 |
<krinkle> |
Synchronized php-1.26wmf12/extensions/WikiEditor/modules/jquery.wikiEditor.toolbar.js: I3e965dda1c4 (duration: 00m 12s) |
[production] |
| 02:27 |
<LocalisationUpdate> |
completed (1.26wmf12) at 2015-07-07 02:27:55+00:00 |
[production] |
| 02:24 |
<l10nupdate> |
Synchronized php-1.26wmf12/cache/l10n: (no message) (duration: 06m 09s) |
[production] |
| 01:12 |
<ori> |
Re-pooled mw1152 at 20:46 UTC, did not log it then. |
[production] |
| 00:41 |
<springle> |
upgrade db1041 trusty |
[production] |
| 00:37 |
<krenair> |
Synchronized php-1.26wmf12/extensions/CentralAuth/includes/CreateLocalAccountJob.php: https://gerrit.wikimedia.org/r/#/c/223211/ (duration: 00m 13s) |
[production] |
|
2015-07-06
§
|
| 23:50 |
<krenair> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/221989/ (duration: 00m 12s) |
[production] |
| 23:49 |
<krenair> |
Synchronized w/static/images/project-logos/mrwikisource.png: https://gerrit.wikimedia.org/r/#/c/221989/ (duration: 00m 13s) |
[production] |
| 23:36 |
<krenair> |
Synchronized wmf-config/abusefilter.php: https://gerrit.wikimedia.org/r/#/c/223179/ - should be labs-only (duration: 00m 12s) |
[production] |
| 23:32 |
<krenair> |
Synchronized README: https://gerrit.wikimedia.org/r/#/c/222941/ - ... (duration: 00m 13s) |
[production] |
| 23:27 |
<krenair> |
Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/221809/ - should be a noop, just doc changes (duration: 00m 13s) |
[production] |
| 23:25 |
<krenair> |
Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/221808/ (duration: 00m 13s) |
[production] |
| 23:17 |
<krenair> |
Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/#/c/223185/ (duration: 00m 12s) |
[production] |
| 23:06 |
<krenair> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/220970/ (duration: 00m 14s) |
[production] |
| 21:46 |
<gwicke> |
restarted cassandra instance on restbase1003; was low on memory and constantly writing small chunks |
[production] |
| 21:30 |
<andrewbogott> |
rebooting labvirt1005, again. Somehow virtualization is turned off again |
[production] |
| 21:12 |
<subbu> |
deployed parsoid version 87a746e6 |
[production] |
| 21:04 |
<ori> |
Synchronized php-1.26wmf12/thumb.php: cdc75debaf: Add Content-Length header to thumb.php error responses (duration: 00m 13s) |
[production] |
| 21:02 |
<mutante> |
purging static-bz URL on varnish ... |
[production] |
| 20:39 |
<akosiaris> |
upload php5_5.3.10-1ubuntu3.19-wmf1 on apt.wikimedia.org/precise-wikimedia |
[production] |
| 20:15 |
<gwicke> |
restart cassandra instance on 1005 |
[production] |
| 20:04 |
<mobrovac> |
restbase restart cassandra on rb1005 |
[production] |
| 19:28 |
<krenair> |
Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/223040/ (duration: 00m 12s) |
[production] |
| 19:11 |
<gwicke> |
reduced compaction throughput from 160 to 100 mb/s across the cassandra cluster via 'nodetool -h <host> setcompactionthroughput 100' |
[production] |
| 18:51 |
<gwicke> |
restarted cassandra on restbase1001 with jdk8, see T104888 |
[production] |
| 18:22 |
<gwicke> |
restarted cassandra on restbase1004 with jdk8 |
[production] |
| 17:54 |
<Jeff_Green> |
authdns-update for new rigel A record |
[production] |
| 17:42 |
<jynus> |
Synchronized wmf-config/db-codfw.php: increase db2029 traffic to normal levels (duration: 00m 12s) |
[production] |
| 17:37 |
<gwicke> |
upgraded restbase1005 to jdk8 |
[production] |
| 17:35 |
<gwicke> |
restarting cassandra instance on restbase1005: out of heap |
[production] |
| 17:10 |
<jynus> |
Synchronized wmf-config/db-codfw.php: repool db2029 again after conf upgrade(2/2) (duration: 00m 11s) |
[production] |
| 17:09 |
<jynus> |
Synchronized wmf-config/db-codfw.php: repool db2029 again after conf upgrade (duration: 00m 11s) |
[production] |
| 16:38 |
<jynus> |
upgrade and restart of db2029 |
[production] |
| 16:35 |
<ori> |
depooled mw1152 |
[production] |
| 15:29 |
<krenair> |
Finished scap: https://gerrit.wikimedia.org/r/#/c/222993/ (duration: 22m 09s) |
[production] |
| 15:21 |
<_joe_> |
repooling mw1152 |
[production] |
| 15:20 |
<_joe_> |
attempting dump-apc on mw1060 |
[production] |
| 15:09 |
<_joe_> |
depooled the HHVM imagescaler again |
[production] |
| 15:07 |
<krenair> |
Started scap: https://gerrit.wikimedia.org/r/#/c/222993/ |
[production] |
| 15:02 |
<krenair> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/222617/ (duration: 00m 12s) |
[production] |
| 14:48 |
<moritzm> |
installed python security updates on analytics*, lab* and virt* |
[production] |
| 14:46 |
<moritzm> |
added python-diskimage-builder 0.1.46-1+wmf1 for jessie-wikimedia on carbon |
[production] |
| 14:43 |
<_joe_> |
depooled the HHVM imagescaler, spitting 503s again. |
[production] |
| 14:18 |
<mobrovac> |
restbase started thinning out parsoid data (local_group_wikipedia_T_parsoid_dataDVIsgzJSne8k) for >= 22 days |
[production] |
| 14:07 |
<YuviPanda> |
restart apache on labcontrol1001 to pick up parser function change |
[production] |