101-150 of 10000 results (48ms)
2019-04-04 ยง
23:52 <bd808@deploy1001> Synchronized php-1.33.0-wmf.23/extensions/LdapAuthentication: SWAT: [[gerrit:501412|Also set an LDAP password policy on Block]] (T168692) (duration: 01m 01s) [production]
23:38 <bd808@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:501393|Add smn and sms to wmgExtraLanguageNames]] (T220118) (duration: 01m 02s) [production]
21:22 <XioNoX> renumber AS58587 to AS10075 in eqsin [production]
21:17 <bblack> DNS deploying https://gerrit.wikimedia.org/r/c/operations/dns/+/500731 which can affect resolution of our CNAME records. If dns-related issues, can revert at will! [production]
21:09 <herron> restarting eqiad ELK stack for security updates [production]
20:45 <marxarelli> promotion of 1.33.0-wmf.24 rolled back to group0 and holding. cc: T206678, T220037 [production]
20:41 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: Revert "group2/group1 wikis to 1.33.0-wmf.24" [production]
20:36 <marxarelli> rolling back again following still high rates of DBTransactionError (avg ~ 800/min) [production]
20:16 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.24 [production]
20:11 <marxarelli> promoting 1.33.0-wmf.24 to all wikis [production]
20:11 <marxarelli> error rates look good after proper syncs and re-deploy. cc: T220037 [production]
20:06 <dduvall@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/Citoid/modules/ve.ui.Citoid.init.js: sync for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Citoid/+/501114 (duration: 00m 58s) [production]
20:04 <dduvall@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/LdapAuthentication/LdapAuthenticationPlugin.php: sync for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/LdapAuthentication/+/500994 (duration: 00m 57s) [production]
20:03 <dduvall@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/LdapAuthentication/LdapAuthenticationHooks.php: sync for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/LdapAuthentication/+/500994 (duration: 00m 58s) [production]
20:02 <dduvall@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/LdapAuthentication/LdapAuthentication.php: sync for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/LdapAuthentication/+/500994 (duration: 00m 58s) [production]
19:58 <dduvall@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/EventBus/includes/JobExecutor.php: syncing JobExecutor changes (duration: 00m 58s) [production]
19:55 <dduvall@deploy1001> Synchronized php: group1 wikis to 1.33.0-wmf.24 (duration: 01m 47s) [production]
19:53 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.33.0-wmf.24 [production]
19:51 <marxarelli> re-deploying to group1 after proper syncs [production]
19:47 <dduvall@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/Citoid/modules/ve.ui.Citoid.init.js: (no justification provided) (duration: 00m 59s) [production]
19:46 <dduvall@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/EventBus/includes/JobExecutor.php: (no justification provided) (duration: 00m 58s) [production]
19:45 <dduvall@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/LdapAuthentication/LdapAuthenticationPlugin.php: (no justification provided) (duration: 00m 58s) [production]
19:44 <dduvall@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/LdapAuthentication/LdapAuthenticationHooks.php: (no justification provided) (duration: 00m 59s) [production]
19:43 <dduvall@deploy1001> Synchronized php-1.33.0-wmf.24/extensions/LdapAuthentication/LdapAuthentication.php: (no justification provided) (duration: 00m 59s) [production]
19:19 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: Revert "group1 wikis to 1.33.0-wmf.24" [production]
19:13 <marxarelli> large spike in DBTransactionError errors. rolling back. cc: T220037 [production]
19:12 <dduvall@deploy1001> Synchronized php: group1 wikis to 1.33.0-wmf.24 (duration: 01m 46s) [production]
19:10 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.33.0-wmf.24 [production]
19:06 <marxarelli> fetch/rebase looks good, incorporates fixes for T220037, T219510. deploying [production]
19:03 <marxarelli> preparing to promote 1.33.0-wmf.24 to group1 [production]
18:33 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable partial blocks on frwiki, plwiki (T219327, T219218) (duration: 00m 58s) [production]
18:23 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable ORES RCFilters on eswikiquote (T219160) (duration: 01m 02s) [production]
18:13 <moritzm> restarted apache on people.wikimedia.org to pick up OpenSSL update [production]
17:59 <bstorm_> stopped postgresql on labsdb1006.eqiad.wmnet and moved the database master functionality (and all rsyncs) to clouddb1003.clouddb-services.eqiad.wmflabs [production]
17:59 <ppchelko@deploy1001> Finished deploy [cpjobqueue/deploy@922cbc0]: Switch to new logging infrastructure T211125 (duration: 04m 03s) [production]
17:55 <ppchelko@deploy1001> Started deploy [cpjobqueue/deploy@922cbc0]: Switch to new logging infrastructure T211125 [production]
17:47 <ppchelko@deploy1001> Finished deploy [changeprop/deploy@f69dc9c]: Switch to new logging infrastructure T211125 (duration: 01m 44s) [production]
17:45 <ppchelko@deploy1001> Started deploy [changeprop/deploy@f69dc9c]: Switch to new logging infrastructure T211125 [production]
17:33 <jynus> stopping replication on dbstore2001:s8 for backup testing T206203 [production]
17:29 <jynus> killing ongoing backup at dbprov2002, stuck [production]
17:28 <gehel@cumin2001> END (ERROR) - Cookbook sre.elasticsearch.rolling-reboot (exit_code=97) [production]
17:10 <gehel@cumin2001> START - Cookbook sre.elasticsearch.rolling-reboot [production]
16:31 <herron> beginning rolling kafka restarts on kafka200[123] for security updates [production]
16:01 <herron> repooling kafka2003 eventbus [production]
15:59 <mutante> wikivoyage-old.org domain has been retired and deactivated (T219867, T81727) [production]
15:56 <herron> depooling kafka2003 for eventbus security updates [production]
15:55 <herron> repooling kafka2002 eventbus [production]
15:52 <herron> depooling kafka2002 for eventbus security updates [production]
15:52 <herron> pooling kafka2001 eventbus [production]
15:42 <herron> depooling kafka2001 for eventbus security updates [production]