4451-4500 of 10000 results (52ms)
2017-06-19 §
07:42 <moritzm> restarting app server canaries to pick up gnutls update [production]
07:13 <marostegui> Reboot ms-be1010 [production]
07:10 <marostegui> Deploy alter table s5 - codfw master - db2023 (and will replicate) so this will generate lag on codfw slaves - T166207 [production]
07:09 <jynus> upgrade, reboot and clear data on pc2006 [production]
07:05 <jynus> upgrade, reboot and clear data on pc2005 [production]
07:03 <jynus@tin> Synchronized wmf-config/db-codfw.php: Depool pc2005 & pc2006 (duration: 00m 41s) [production]
06:58 <moritzm> installing gnutls security updates [production]
06:38 <marostegui> Deploy alter table s2 - labsdb1001 - T166205 [production]
06:37 <jynus> force learning cycle to db1046 controller T166141 [production]
06:23 <marostegui> Deploy alter table on s2 - db1021 - T166205 [production]
06:18 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1021 - T166205 (duration: 00m 41s) [production]
04:21 <reedy@tin> Synchronized composer.lock: update (duration: 00m 41s) [production]
04:20 <reedy@tin> Synchronized composer.json: update (duration: 00m 41s) [production]
04:19 <reedy@tin> Synchronized multiversion/vendor/: Update! (duration: 01m 05s) [production]
04:05 <reedy@tin> Synchronized wmf-config/CommonSettings.php: Fix comments minor code style (duration: 00m 42s) [production]
02:26 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Jun 19 02:26:06 UTC 2017 (duration 6m 8s) [production]
02:19 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.5) (duration: 07m 04s) [production]
2017-06-18 §
02:25 <l10nupdate@tin> ResourceLoader cache refresh completed at Sun Jun 18 02:25:55 UTC 2017 (duration 6m 8s) [production]
02:19 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.5) (duration: 07m 27s) [production]
2017-06-17 §
19:30 <ebernhardson> restarting elasticsearch on relforge to pick up new vrsion of ltr-query [production]
16:51 <volans> restarted pdfrender on scb200[2,4] T159922 [production]
15:26 <jynus> rebuild pc2004's (depooled) data from scratch [production]
02:29 <l10nupdate@tin> ResourceLoader cache refresh completed at Sat Jun 17 02:29:51 UTC 2017 (duration 6m 8s) [production]
02:23 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.5) (duration: 07m 09s) [production]
2017-06-16 §
19:54 <Reedy> disabled cluster 2fa for Chrissymad for T168064 (confirmed by email) [production]
19:26 <ejegg> re-enabled paypal audit download and parse job [production]
19:13 <ebernhardson> restarting elasticesarch on relforge to pick up new ltr-query plugin version [production]
18:14 <mutante> ms-be1001: did not change config, tried again, now detected 13 drives again, coming back [production]
18:10 <mutante> ms-be1001 - The following VDs are missing: 09 [production]
18:08 <mutante> ms-be1001 - powercycling crashed server - "[14076481.245487] general protection fault: 0000 [#4] SMP [production]
13:36 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove comments from db1018 current status - T166205 (duration: 00m 41s) [production]
13:26 <twentyafterfour> fixed phabricator "upgrade database" error. [production]
13:20 <twentyafterfour> fixing phab database migrations [production]
13:01 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1091 after performance testing (duration: 00m 41s) [production]
10:18 <jynus> running analyze on db1091 (depooled), may create lag [production]
10:11 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1091 for performance testing (duration: 00m 42s) [production]
09:52 <moritzm> installing guile security updates [production]
09:13 <moritzm> re-enabled puppet on mw2129 (no reason was given why it was disabled( [production]
08:50 <jynus> bringing down pc1005 and pc1006 for maintenance T167567 [production]
08:40 <jynus@tin> Synchronized wmf-config/db-codfw.php: Add db1099 and db1001 hosts to config (duration: 00m 41s) [production]
08:23 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Switchover pc1005 and pc1006 to db1099 and db1001 (duration: 00m 45s) [production]
08:20 <jynus> about to swithover pc1005 and pc1006 to db1099 and db1001 [production]
05:45 <ebernhardson> increase enwiki_content replicas on codfw from 2 to 3 to match eqiad [production]
02:37 <l10nupdate@tin> ResourceLoader cache refresh completed at Fri Jun 16 02:37:05 UTC 2017 (duration 6m 25s) [production]
02:30 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.5) (duration: 07m 02s) [production]
2017-06-15 §
23:37 <mutante> added stretch support for jenkins (https://gerrit.wikimedia.org/r/#/c/359227/, https://gerrit.wikimedia.org/r/#/c/359356/) | 'reprepro copy stretch-wikimedia jessie-wikimedia jenkins' to make .deb available on stretch | releases1001 now running jenkins , icinga recovered | (hashar) (T164030) [production]
23:30 <mutante> APT - reprepro copy strech-wikimedia jessie-wikimedia jenkins (copy existing jenkins package to stretch, it can be used on both) [production]
23:18 <ebernhardson@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: T166408: Remove dead config variable MinervaPrintStyles (duration: 00m 41s) [production]
23:15 <ebernhardson@tin> Finished scap: wmf-config Scap: T162276: Enable crossproject search (duration: 03m 37s) [production]
23:11 <ebernhardson@tin> Started scap: wmf-config Scap: T162276: Enable crossproject search [production]