3401-3450 of 10000 results (38ms)
2016-06-23 §
13:30 <jynus> db1059 backup and reimage [production]
13:28 <awight> update SmashPig from c0cc2a1a6062ad8d114473ea1a444786a0d50833 to a435adeb130217bda8b95d3c5c6331ace8ad1228 [production]
13:16 <jynus> running scap pool on mw1301 [production]
13:13 <mobrovac> restarting zotero on sca, 6g mem [production]
13:13 <jynus> running scap pool on mw1300 [production]
13:11 <mobrovac> citoid deploying 0129ab0b [production]
13:11 <elukey> purged some puppet output logs on compiler02.puppet3-diffs.eqiad.wmflabs to free space (disk full) [production]
13:09 <moritzm> depooled jessie image scaler (mw1291) again, works fine, to be permanently pooled on Monday [production]
12:49 <moritzm> pooling new jessie image scaler mw1291 for short production smoke testing [production]
12:35 <awight> update SmashPig from f7d65c54bed3ff9c478b0dbcaa1b2d27cc665ace to c0cc2a1a6062ad8d114473ea1a444786a0d50833 [production]
12:18 <awight> update SmashPig from 90757321a3bfa1045202e06e3dd1960a0043493a to f7d65c54bed3ff9c478b0dbcaa1b2d27cc665ace [production]
12:07 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1059; Repool db1061 & db1062; increase weight of db1068 (duration: 00m 39s) [production]
11:32 <gehel> rolling restart of elasticsearch10(01|30|08|36|13|40) to activate new masters [production]
10:13 <andrewbogott> restarting rabbitmq-server on labcontrol1001 (random debugging attempt for T138106) [production]
09:49 <godog> reimage ms-be202[567] with incorrect raid settings [production]
09:11 <jynus> syncing etherpadlite.store (m1) on db2010, which had 2 bad chunks [production]
08:39 <mobrovac> change-prop restarting on scb to pick up ores rules https://gerrit.wikimedia.org/r/295576 [production]
08:06 <mobrovac> change-prop deploying 45db4f84827 [production]
06:59 <moritzm> installing spice security updates [production]
02:47 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu Jun 23 02:47:59 UTC 2016 (duration 6m 44s) [production]
02:41 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.7) (duration: 07m 05s) [production]
02:26 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.6) (duration: 11m 19s) [production]
2016-06-22 §
23:24 <maxsem@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/295560/ (duration: 00m 25s) [production]
23:23 <maxsem@tin> Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/295560/ (duration: 00m 24s) [production]
23:23 <maxsem@tin> Synchronized dblists/mobilemainpagelegacy.dblist: https://gerrit.wikimedia.org/r/#/c/295560/ (duration: 00m 24s) [production]
23:14 <maxsem@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/294247/ (duration: 00m 24s) [production]
23:09 <maxsem@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/295558/ (duration: 00m 40s) [production]
22:25 <ori> Ran hacked maintain-replicas.pl on labsdb100[13] for T135029 [production]
21:06 <bblack> cache perf: start deploy of -autocorking (probably last experiment I can squeeze in today) [production]
20:59 <Dereckson> Run namespaceDupes.php on ptwikinews (T138230) and frwikinews (T138442) [production]
20:33 <mdholloway> mobileapps: finished deploying 8046ee2 [production]
20:26 <yurik> deployed & restarted tilerator https://gerrit.wikimedia.org/r/#/c/295447/ [production]
20:25 <mdholloway> starting mobileapps deployment [production]
20:20 <Reedy> created tmplog_begin_devices on tmplog_end_devices on testwiki.cn_template_log [production]
20:18 <yurik> deployed & restarted kartotherian https://gerrit.wikimedia.org/r/#/c/295449/ [production]
19:32 <bblack> start rollout of first batch of cache sysctl stuff (un-mysterious + disable prequeue timestamps) [production]
19:29 <jynus> archiving and dropping reviewdb on m1 shard [production]
19:06 <thcipriani@tin> rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.28.0-wmf.7 [production]
18:46 <jynus> shutting down and reimaging db1001 [production]
18:20 <papaul> ms-be202[3-7] - signing puppet certs, salt-key, initial run [production]
17:23 <akosiaris> restart apache on ununpentium for m1 migration. Hosts RT, just did it for good measure [production]
17:21 <akosiaris> restarted bacula-director on helium [production]
17:15 <jynus> killing puppet, rt, librenms user connections on db1001 [production]
17:10 <jynus> failovered m1-master from db1001 to db1016 [production]
16:20 <gehel> new elasticsearch servers elastic1032-1047 are configured and have joined the eqiad cluster [production]
15:26 <thcipriani@tin> Synchronized php-1.28.0-wmf.6/extensions/OATHAuth: SWAT: [[gerrit:295510|Fixup qrcode-generating js, to stop race condition.]] (duration: 00m 33s) [production]
15:23 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:295514|Improve style]] (duration: 00m 33s) [production]
15:18 <thcipriani@tin> Synchronized php-1.28.0-wmf.7/extensions/OATHAuth: SWAT: [[gerrit:295511|Fixup qrcode-generating js, to stop race condition.]] (duration: 00m 27s) [production]
15:13 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:295478|Add www.wpc.ncep.noaa.gov to wgCopyUploadsDomains]] (duration: 00m 54s) [production]
15:01 <elukey> rebooting bohrium.eqiad.wmnet (running piwik) for kernel upgrades [production]