4701-4750 of 10000 results (36ms)
2016-08-11 §
17:48 <ema> switched LVS schedulers for text, upload, maps and misc port 80 to source hash scheduling T108827 [production]
17:37 <ema> applying https://gerrit.wikimedia.org/r/#/c/297418/ to eqiad load balancers [production]
17:16 <ema> applying https://gerrit.wikimedia.org/r/#/c/297418/ to codfw load balancers [production]
17:02 <Amir1> restarting celery-ores-worker service in scb[12]00[12] T142361 [production]
17:00 <ema> applying https://gerrit.wikimedia.org/r/#/c/297418/ to ulsfo load balancers [production]
16:36 <ema> applying https://gerrit.wikimedia.org/r/#/c/297418/ to esams load balancers [production]
15:51 <ema> disabling puppet on lvs hosts to test https://gerrit.wikimedia.org/r/#/c/297418/ [production]
14:20 <jynus> disabling puppet on db2001-2009; preparing for decomission [production]
13:53 <chasemp> restarting nodepool to test stats collection [production]
13:02 <elukey> restarting cassandra on aqs100[123] for jvm upgrades [production]
10:26 <elukey> restarting statsv on hafnium (process stuck after kafka brokers restart) [production]
10:25 <elukey> kafka restarted on kafka200[12] for jvm upgrades [production]
09:34 <jynus> removing leftovers of unmaintained skrillex tool from mira and tin [production]
09:17 <elukey> restarting statsv on hafnium (blocked due to analytics kafka brokers restarts) [production]
09:01 <elukey> upgrading httpd to 2.4.10-10+deb8u6+wmf2 on mw126[234] [production]
08:50 <elukey> uploaded apache2 2.4.10-10+deb8u6+wmf2 to reprepro [production]
08:18 <elukey> restarting kafka on the kafka analytics cluster for jvm upgrades [production]
07:38 <elukey> increaseing traffic weight to 30 for mw1261 (current: 5) [production]
02:56 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu Aug 11 02:56:27 UTC 2016 (duration 6m 54s) [production]
02:49 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.14) (duration: 10m 24s) [production]
02:27 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.13) (duration: 11m 12s) [production]
00:41 <aaron@tin> Synchronized php-1.28.0-wmf.14/maintenance/purgeChangedFiles.php: 91baa668219d8a77b1e300c257e969b04b4b1e3d (duration: 00m 47s) [production]
00:39 <aaron@tin> Synchronized php-1.28.0-wmf.13/maintenance/purgeChangedFiles.php: 4fe4f803541c4b21c5a118eab426d78d6c0c607b (duration: 00m 50s) [production]
00:10 <reedy@tin> Synchronized wikiversions.json: noop for mira (duration: 00m 49s) [production]
2016-08-10 §
23:59 <AaronSchulz> Running purgeChangedFiles.php on all wikis on a terbium screen (T142638) [production]
23:46 <reedy@tin> rebuilt wikiversions.php and synchronized wikiversions files: Reinstate .14 as T142638 is fixed [production]
23:44 <mutante> mw2086 - removing node from cluster failed - backend error, request requires authentication [production]
23:40 <reedy@tin> Synchronized php-1.28.0-wmf.14/extensions/VipsScaler: Remove old broken config causing T142638 (duration: 00m 50s) [production]
23:39 <yuvipanda> restarted rabbitmq on labcontrol1001 [production]
23:22 <mutante> connected to mw2086.mgmt (which icinga said was down since a couple hours). i saw it booting up..it came back but i did not powercyle or reboot, just view console [production]
23:17 <reedy@tin> rebuilt wikiversions.php and synchronized wikiversions files: Revert to .13 to attempt to fix T142638 [production]
22:36 <andrewbogott> restarting rabbitmq-server on labvirt1001 [production]
21:42 <mutante> added tbayer(HaeB) to wmf LDAP group [production]
21:30 <eileen> updated civicrm from 024a7ef2badb02743659ffed9856571d96604fd2 to cad2c59081e53b877f51be4c853f307f3c4d4949 [production]
20:21 <Amir1> deploygin 7aad8e9 for ores in all nodes [production]
20:18 <Amir1> deploygin 7aad8e9 for ores in canary (scb2001.codfw.wmnet) [production]
20:17 <mattflaschen@tin> Synchronized wmf-config/CommonSettings-labs.php: Re-enable thank-you-edit (milestone notifications for 1st, 10th, 100th, etc. edits) on Beta Cluster (duration: 02m 45s) [production]
20:11 <subbu> finished deploying parsoid sha 4de49e26 [production]
20:07 <subbu> finished syncing code; restarted parsoid on wtp1001 as canary [production]
20:05 <subbu> starting parsoid deploy [production]
19:34 <twentyafterfour@tin> rebuilt wikiversions.php and synchronized wikiversions files: (no message) [production]
19:31 <twentyafterfour> sync-wikiversions failed for mw2086.codfw.wmnet [production]
19:30 <twentyafterfour@tin> rebuilt wikiversions.php and synchronized wikiversions files: (no message) [production]
17:15 <otto@palladium> conftool action : set/pooled=yes; selector: kafka1002.eqiad.wmnet [production]
17:14 <otto@palladium> conftool action : set/pooled=no; selector: kafka1002.eqiad.wmnet [production]
17:14 <otto@palladium> conftool action : set/pooled=yes; selector: kafka1001.eqiad.wmnet [production]
17:12 <otto@palladium> conftool action : set/pooled=no; selector: kafka1001.eqiad.wmnet [production]
17:12 <ottomata> restarting eventbus in eqiad to apply error_output change [production]
16:13 <hoo@tin> Synchronized wmf-config/InitialiseSettings.php: Enable the ArticlePlaceholder on knwiki (T142468) (duration: 02m 55s) [production]
16:08 <hoo@tin> Synchronized wmf-config/InitialiseSettings.php: Enable the ArticlePlaceholder on cywiki (T140725) (duration: 02m 49s) [production]