451-500 of 10000 results (27ms)
2015-06-15 §
10:44 <_joe_> disabled hot threads logging, ganglia on es nodes [production]
10:44 <manybubbles> started Elasticsearch on all elasticsearch nodes [production]
10:38 <manybubbles> stopping all elasticsearch servers - going for a full cluster resstart. [production]
10:11 <manybubbles> restarting elasticsearch on elasticsearch1021 - that one is in a gc death spiral [production]
09:26 <oblivian> Synchronized wmf-config/PoolCounterSettings-common.php: temporarily throttle down cirrussearch (duration: 00m 13s) [production]
09:12 <oblivian> Synchronized wmf-config/PoolCounterSettings-common.php: temporarily throttle down cirrussearch (duration: 00m 13s) [production]
07:35 <_joe_> attempting a fast restart of elastic1020 [production]
07:21 <ori> Synchronized php-1.26wmf9/extensions/CirrusSearch/includes/Util.php: I504dac0c3: Add missing 'use \\Status;' to includes/Util.php (duration: 00m 13s) [production]
04:56 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Jun 15 04:56:39 UTC 2015 (duration 56m 38s) [production]
03:31 <springle> Synchronized wmf-config/db-eqiad.php: depool db1057 (duration: 00m 12s) [production]
02:23 <LocalisationUpdate> completed (1.26wmf9) at 2015-06-15 02:22:56+00:00 [production]
02:19 <l10nupdate> Synchronized php-1.26wmf9/cache/l10n: (no message) (duration: 05m 46s) [production]
2015-06-14 §
10:39 <YuviPanda> running du -d 2 on /srv/project in a screen sesssion on labstore1001 [production]
04:33 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Jun 14 04:33:20 UTC 2015 (duration 33m 19s) [production]
02:42 <reedy> Synchronized wmf-config/extension-list: noop (duration: 00m 13s) [production]
02:40 <krenair> Synchronized wmf-config/squid-labs.php: sync random labs-only file to test per irc (duration: 00m 13s) [production]
02:21 <LocalisationUpdate> completed (1.26wmf9) at 2015-06-14 02:21:28+00:00 [production]
02:18 <l10nupdate> Synchronized php-1.26wmf9/cache/l10n: (no message) (duration: 05m 47s) [production]
2015-06-13 §
19:30 <bblack> repooled cp1071, cp3040 [production]
18:53 <bblack> rebooting cp1071, cp3040 to look at BIOS-level things (depooled, icinga-downed) [production]
17:08 <krinkle> Synchronized php-1.26wmf9/extensions/WikimediaEvents: T101806 (duration: 00m 12s) [production]
15:47 <paravoid> labstore1001: stopping manage-nfs-volumes daemon [production]
04:42 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Jun 13 04:41:57 UTC 2015 (duration 41m 56s) [production]
03:51 <Krinkle> Running deleteEqualMessages.php for sawiki (T45917) [production]
03:49 <Krinkle> Running deleteEqualMessages.php for cewiki (T45917) [production]
02:21 <LocalisationUpdate> completed (1.26wmf9) at 2015-06-13 02:20:58+00:00 [production]
02:18 <l10nupdate> Synchronized php-1.26wmf9/cache/l10n: (no message) (duration: 05m 19s) [production]
00:17 <gwicke> restarted cassandra on restbase1001 [production]
00:13 <gwicke> restarted cassandra on restbase1002 [production]
2015-06-12 §
22:57 <ejegg> rolled back SmashPig on listener from 15acdafef9d9682c417632e5ac5a5f2e5380f92e to e1e925c9fc2a60c1e14ef01d8b653dc09512f51f [production]
22:40 <ejegg> updated SmashPig on listener from e1e925c9fc2a60c1e14ef01d8b653dc09512f51f to 15acdafef9d9682c417632e5ac5a5f2e5380f92e [production]
22:24 <godog> upgrade and bounce carbon daemons on graphite2001 to investigate T101572 [production]
21:16 <ori> Synchronized wmf-config/InitialiseSettings.php: I3694489ba: wgCanonicalServer->https for new HTTPS domains (duration: 00m 14s) [production]
20:33 <krenair> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/217878/1 (duration: 00m 13s) [production]
20:32 <krenair> Synchronized w/static/images/project-logos/dawiki-200k.png: https://gerrit.wikimedia.org/r/#/c/217878/1 (duration: 00m 16s) [production]
20:15 <krenair> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/217670/ (duration: 00m 12s) [production]
19:28 <ejegg> updated SmashPig on payments-listener from f9c3eaa99fa0fe8ef098d0fc876091d3676aa039 to 5a463400bc74706ba7bf6256cd0101014e792acb [production]
19:28 <ejegg> updated SmashPig on payments-listener ccepting New Patients: [production]
18:47 <ejegg> updated SmashPig on payments-listener from 7fed22ad933a6d3e371d60dfc6f8fdd0f9131510 to f9c3eaa99fa0fe8ef098d0fc876091d3676aa039 [production]
18:45 <faidon> Synchronized wmf-config/InitialiseSettings.php: remove wmgHTTPSBlacklistCountries (duration: 00m 12s) [production]
18:45 <faidon> Synchronized wmf-config/CommonSettings.php: remove CanIPUseHTTPS hook (duration: 00m 13s) [production]
17:39 <moritzm> updated cerium, xenon and praseodymium to 3.19 kernel [production]
17:08 <ejegg> enabled queue consumer [production]
17:08 <ejegg> updated crm from d13aaa4e9e937b0b1ae1f5de61ea7ff1f316d58f to bd8a00196071ddd04efbff7b30567dd9357c9000 [production]
16:53 <ejegg> disabled donations queue consumer [production]
15:52 <faidon> Synchronized wmf-config/CommonSettings.php: hide prefershttps user pref (duration: 00m 13s) [production]
15:40 <faidon> Synchronized docroot/search.wikimedia.org/index.php: unbreak search.wikimedia.org due to HTTPS (duration: 00m 12s) [production]
15:27 <jynus> mysql load issues on labsdb1003, investigating [production]
13:39 <moritzm> updated etcd* to 3.19 kernel [production]
12:11 <jynus> restarting mariadb at labsdb1003 [production]