151-200 of 10000 results (28ms)
2016-03-23 §
11:54 <godog> swift eqiad-prod ms-be1020 / ms-be1021 to weight 3500 [production]
10:59 <jynus> stopping and restarting db1015 for upgrade and clone to db1077 [production]
10:44 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool db1015, increase weight of db1044 (duration: 00m 25s) [production]
10:38 <godog> depool restbase1003 / restbase1004 prior to deprovisioning the hardware [production]
10:29 <moritzm> installing various security updates on mediawiki codfw servers (along with HHVM restarts): graphite2, libldap, pixman, sqlite, pygments, gnutls26 (already running fine on canaries since yesterday) [production]
09:21 <jynus> start mysql on es2019 at es2018-bin.000044:287914983 [production]
08:38 <jynus> starting mysql at es2019 [production]
08:37 <jynus@tin> Synchronized wmf-config/db-codfw.php: Add db2008 (x1) depooled, depool es2019 (duration: 00m 26s) [production]
08:30 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1044, pool db1075 (duration: 00m 25s) [production]
08:27 <jynus@tin> Synchronized wmf-config/db-codfw.php: Add db1075 (duration: 00m 40s) [production]
07:59 <jynus> powercycling es2019 - it was down [production]
07:19 <_joe_> progressively activating cross-dc replica and encryption between the jobqueue redises [production]
03:34 <mutante> tin - re-arm keyholder [production]
03:26 <mutante> tin - restart keyholder - re: < twentyafterfour> 1 mismatch. I guess that keyholder-proxy needs to be restarted on tin [production]
03:12 <l10nupdate@tin> ResourceLoader cache refresh completed at Wed Mar 23 03:12:07 UTC 2016 (duration 9m 20s) [production]
03:02 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.18) (duration: 17m 31s) [production]
02:27 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.17) (duration: 11m 18s) [production]
2016-03-22 §
23:47 <maxsem@tin> Synchronized php-1.27.0-wmf.18/extensions/WikimediaEvents/: https://gerrit.wikimedia.org/r/#/c/279066/ (duration: 00m 27s) [production]
23:46 <maxsem@tin> Synchronized php-1.27.0-wmf.17/extensions/WikimediaEvents/: https://gerrit.wikimedia.org/r/#/c/279066/ (duration: 00m 28s) [production]
23:40 <maxsem@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT x 3 (duration: 00m 26s) [production]
23:35 <maxsem@tin> Synchronized portals: (no message) (duration: 00m 25s) [production]
23:35 <maxsem@tin> Synchronized portals/prod/wikipedia.org/assets: (no message) (duration: 00m 26s) [production]
23:26 <MaxSem> ran echo 'https://test.wikidata.org/static/favicon/testwikidata.ico' | mwscript purgeList.php [production]
23:23 <maxsem@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/278334/ (duration: 00m 27s) [production]
23:23 <maxsem@tin> Synchronized static/images/project-logos: https://gerrit.wikimedia.org/r/#/c/278334/ (duration: 00m 26s) [production]
23:20 <maxsem@tin> Synchronized static/favicon/testwikidata.ico: https://gerrit.wikimedia.org/r/#/c/278321/ (duration: 00m 31s) [production]
23:15 <mutante> mira - restarting and re-arming keyholder [production]
23:14 <mutante> tin - restarting and re-arming keyholder [production]
23:04 <mutante> re-arm keyholder on tin for new phab key [production]
23:02 <mutante> mira re-arm keyholder for new phab key [production]
22:56 <aaron@tin> Synchronized wmf-config: Lower "max lag" and $wgAPIMaxLagThreshold to 8/6 (duration: 00m 29s) [production]
22:52 <aaron@tin> Synchronized wmf-config/jobqueue-eqiad.php: Bump timeout to 300ms (duration: 00m 29s) [production]
22:50 <aaron@tin> Synchronized wmf-config/filebackend-production.php: comment tweak (duration: 00m 35s) [production]
22:50 <mutante> replacing phab deploy key with a new one [production]
21:41 <hoo> Ran sync-common on mw1139 (which missed two deploys) [production]
21:28 <bd808> Restarted logstash process on logstash1002; dead from OOM since 2016-03-18T11:47:12 [production]
21:27 <hoo@tin> Synchronized php-1.27.0-wmf.18/extensions/Wikidata: Update Wikibase: Fix add qualifier link not getting disabled (duration: 03m 43s) [production]
21:23 <hoo@tin> Synchronized php-1.27.0-wmf.17/extensions/Wikidata: Update Wikibase: Fix add qualifier link not getting disabled (duration: 07m 30s) [production]
21:01 <mutante> mira - arm keyholder [production]
20:59 <mutante> tin - arm keyholder with deployment keys [production]
20:47 <urandom> Stopping restbase2004.codfw.wmnet for offline sstablescrub : T130254 [production]
20:46 <urandom> decommissioning restbase1004-a.eqiad.wmnet : T125842 [production]
20:45 <urandom> Rolling restart of RESTBase Cassandra cluster complete : T130393, T128787 [production]
20:25 <thcipriani@tin> rebuilt wikiversions.php and synchronized wikiversions files: group0 to 1.27.0-wmf.18 [production]
20:22 <gehel> deployed latest version of WDQS [production]
20:18 <thcipriani@tin> Finished scap: testwiki to php-1.27.0-wmf.18 and rebuild l10n cache (duration: 46m 09s) [production]
19:53 <urandom> Rolling restart of RESTBase Cassandra cluster : T130393, T128787 [production]
19:32 <thcipriani@tin> Started scap: testwiki to php-1.27.0-wmf.18 and rebuild l10n cache [production]
19:00 <mutante> mw1125 - powercycle [production]
18:41 <urandom> Aborting restart of Cassandra on maps cluster : T130393, T128787 [production]