301-350 of 10000 results (44ms)
2018-06-25 §
14:13 <gehel> banning and depooling elastic1042 (high response times) [production]
14:01 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Fix ORES config, part II - T197633 (duration: 00m 57s) [production]
13:59 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Whitelist two Indian government websites - T197944 (duration: 00m 57s) [production]
13:57 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Autoconfirmed should require 10 edits&4 days on zhwikivoyage - T198006 (duration: 00m 56s) [production]
13:54 <hashar> mwscript namespaceDupes.php --wiki=zhwikivoyage --fix | T198007 [production]
13:53 <elukey> merging jmxtrans and kafkatee's submodules to operations/puppet - part 1 (moving them under environments/production) [production]
13:51 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Autoconfirmed should require 10 edits&4 days on zhwikivoyage - T198006 (duration: 00m 56s) [production]
13:50 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Add namespace aliases to zhwikivoyage - T198007 (duration: 00m 56s) [production]
13:37 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Remove BN alias for NS_USER on dewikivoyage - T196905 (duration: 00m 57s) [production]
13:34 <hashar@deploy1001> Synchronized php-1.32.0-wmf.8/extensions/Echo: Remove masterPos from the job specification - T192945 (duration: 00m 58s) [production]
13:32 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Fix ORES config - T197633 (duration: 00m 58s) [production]
13:27 <hashar@deploy1001> Synchronized dblists/commonsuploads.dblist: Add wikimania2018wiki to commonsupload.dblist | T197714 (duration: 00m 56s) [production]
13:12 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable logging for Schema:CitationUsage at 1% | T191086 (duration: 00m 57s) [production]
13:05 <marostegui> Deploy schema change on db2035 (codfw primary master for s2) with replication, this will generate lag on s2 codfw T191316 T192926 T89737 T195193 [production]
13:05 <godog> refresh cassandra certificates for 'restbase' cluster [production]
13:00 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1106 after alter table (duration: 00m 57s) [production]
12:29 <kartik@deploy1001> Finished deploy [cxserver/deploy@cc6dc61]: Update cxserver to ece5e7a (T191874, T196354, T195768, T195768) (duration: 03m 33s) [production]
12:26 <kartik@deploy1001> Started deploy [cxserver/deploy@cc6dc61]: Update cxserver to ece5e7a (T191874, T196354, T195768, T195768) [production]
12:04 <mobrovac> Restbase: Add the "headers" field to Cassandra schemae in production for T197789 [production]
12:00 <mobrovac@deploy1001> Finished deploy [restbase/deploy@f521e7e] (dev-cluster): Dev cluster: Include pagelanguage in title_revisions (duration: 04m 05s) [production]
12:00 <gehel> repooling wdqs1005 it has catched up on updates - T198042 [production]
11:56 <mobrovac@deploy1001> Started deploy [restbase/deploy@f521e7e] (dev-cluster): Dev cluster: Include pagelanguage in title_revisions [production]
10:32 <akosiaris> increase CPU count for proton machines from 2 to 10. T197862 [production]
08:22 <marostegui> Deploy schema change on db1106 with replication, this will generate lag on s1 labs T191316 T192926 T89737 T195193 [production]
08:16 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1106 for alter table (duration: 00m 56s) [production]
08:16 <marostegui> Stop replication on db1106 to change triggers for db1124 and db1095 - T192926 [production]
08:11 <gehel> rolling restart of wdqs to enable async logging - T198051 [production]
08:11 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1080 after alter table (duration: 00m 58s) [production]
07:46 <gehel> depooling wdqs1005 to allow it to catch up on updates - T198042 [production]
07:33 <moritzm> slow rollout of most of the remaining debmonitor client installations [production]
05:43 <marostegui> Deploy schema change on db1080 T191316 T192926 T89737 T195193 [production]
05:41 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1080 for alter table (duration: 01m 17s) [production]
02:39 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.999) (duration: 07m 37s) [production]
02:21 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.8) (duration: 08m 40s) [production]
2018-06-24 §
22:26 <gehel> restarting wdqs1004 - T198042 [production]
21:51 <gehel> restarting wdqs1005 after taking a few thread dumps - T198042 [production]
19:41 <gehel> restart blazegraph on wdqs1004 [production]
16:09 <godog> restart wdqs-updater on wdqs1005 [production]
15:40 <godog> restart wdqs-blazegraph on wdqs1005 [production]
15:40 <jynus> restart graphite2001 [production]
15:35 <godog> systemctl restart wdqs-blazegraph on wdqs1004 [production]
15:33 <godog> restart-wdqs on wdqs1004 [production]
2018-06-22 §
19:54 <jynus> applying transaction manually on dbstore1002 due to weird bug with savepoint on tokudb+image table [production]
2018-06-21 §
21:38 <ebernhardson> banned elastic1036 from search cluster, waited for all load to shift away, and unbanned [production]
16:38 <ejegg> updated CiviCRM from 349d43eb8b to e2c8ddd70e [production]
13:28 <vgutierrez> Bump AES128-SHA pageview replacement to 4% [production]
12:49 <chasemp> remove labvirt1019 canary to start debug of network [production]
11:08 <hashar> Refreshed operations-puppet-tests-docker jenkins job to a new Docker container build that includes isc-dhcp-server | https://gerrit.wikimedia.org/r/c/integration/config/+/441367 [production]
02:51 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.999) (duration: 07m 40s) [production]
02:33 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.8) (duration: 14m 12s) [production]