2851-2900 of 10000 results (31ms)
2015-10-08 §
00:51 <mutante> mw1160 - was oom-killer (convert, mw job 31497) [production]
00:50 <twentyafterfour> phabricator maintenance/upgrade. Expect 10 minutes downtime [production]
00:44 <mutante> powercycled mw1160 - looks like broken hardware or cable (ata1: lost interrupt) [production]
00:08 <krenair@tin> Synchronized w/static/apple-touch/commons.png: https://gerrit.wikimedia.org/r/#/c/243685/ (duration: 00m 17s) [production]
2015-10-07 §
23:36 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/242842/ (duration: 00m 17s) [production]
23:34 <ori@tin> Synchronized wmf-config: I924d8e19e17: Make the redis cache configuration multi-DC-ready (T111575) (duration: 00m 17s) [production]
23:22 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/244177/ (duration: 00m 17s) [production]
23:18 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/244176/ (duration: 00m 17s) [production]
23:08 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/243837/ (duration: 00m 17s) [production]
21:25 <ori@tin> Synchronized wmf-config/CommonSettings.php: Revert "Don't use nutcracker on wikitech" (duration: 00m 16s) [production]
20:56 <mutante> applied firewalling on IRCd server, rc bot still working fine, all public IRC ports as before [production]
20:25 <mutante> argon: installing package upgrades [production]
20:24 <mutante> resetting drac on argon [production]
20:14 <ori@tin> Synchronized wmf-config/session.php: Ie25c368a: Switch mw1017 to use DC-specific redis cluster names (duration: 00m 17s) [production]
19:50 <urandom> RESTBase deploy complete [production]
19:41 <urandom> doing full deploy of c20e6336 to RESTBase [production]
19:29 <urandom> canary deploy to restbase1001.eqiad complete [production]
19:14 <urandom> deploying c20e6336 to canary node restbase1001.eqiad [production]
19:09 <twentyafterfour@tin> rebuilt wikiversions.cdb and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.2 [production]
18:56 <twentyafterfour@tin> Synchronized wmf-config/CommonSettings.php: fix undefined variable warning that has been spamming logs (duration: 00m 17s) [production]
16:57 <wikibugs> Updated channels.yaml to: c78efa6e621b316c26fca060661f59558e8bafa5 Merge "Also exclude TCB-Team- from #wikimedia-fundraising" [tools.wikibugs]
16:39 <mutante> powercycling analytics1035 [production]
16:32 <joal> Started cassandra load jobs from 2015-10-01 [analytics]
16:14 <mobrovac> citoid deploying ec149fd5 [production]
16:02 <marktraceur@tin> Synchronized wmf-config/: Adding new config variable for uploads to Commons (duration: 00m 17s) [production]
15:49 <bd808> Updated cherry-pick of https://gerrit.wikimedia.org/r/#/c/241984 on deployment-puppetmaster and forced puppet run on deployment-logstash2 [releng]
15:48 <bd808> Made [LOCAL HACK] commit on deployment-puppetmaster for l10nupdate cache location change [releng]
15:45 <bd808> Uncommited local change to /var/lib/git/operations/puppet/modules/scap/files/l10nupdate-1 on deployment-puppetmaster. Looks like something Krenair was working on. [releng]
15:44 <catrope@tin> Synchronized php-1.27.0-wmf.1/extensions/MobileFrontend: SWAT (duration: 00m 17s) [production]
15:44 <catrope@tin> Synchronized php-1.27.0-wmf.2/extensions/Echo: SWAT (duration: 00m 18s) [production]
15:44 <catrope@tin> Synchronized php-1.27.0-wmf.1/extensions/Echo: SWAT (duration: 00m 17s) [production]
15:13 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1051 after maintenance (duration: 00m 17s) [production]
15:03 <krenair@tin> Synchronized visualeditor-default.dblist: https://gerrit.wikimedia.org/r/#/c/242041/ (duration: 00m 17s) [production]
15:02 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/242041/ (duration: 00m 18s) [production]
14:59 <mobrovac> AQS restarting restbase on aqs100x [production]
13:57 <hashar> fixing up apt on deployment-cache-parsoid04 as well as salt install https://phabricator.wikimedia.org/T114755 [releng]
13:52 <hashar> Upgrading packages oon deployment-cache-{text04,mobile04,upload04} , downgrading salt-common and salt-minion in the process ( https://phabricator.wikimedia.org/T114755 ) [releng]
13:43 <hashar> Cleaning up <tt>http://debian.saltstack.com/debian/ jessie-saltstack/main</tt> from some beta-cluster instances | https://phabricator.wikimedia.org/T114755 [releng]
13:00 <godog> decomission restbase-test2001 and reimage [production]
12:27 <paravoid> salt-rm'ing /var/lib/apt/lists/ubuntu.wikimedia.org_ubuntu_dists_trusty_main_i18n_Translation-en%5fUS [production]
10:55 <godog> reenable puppet on restbase / maps-test / aqs [production]
10:10 <_joe_> depooling cp1059 from pybal, varnish [production]
10:09 <hashar> deleted angry-caching-proxy.integration.eqiad.wmflabs | was an experiment no more use for it [releng]
08:44 <godog> disable puppet on restbase, maps, aqs before merging https://gerrit.wikimedia.org/r/#/c/243127 [production]
08:05 <moritzm> installed spice security updates on labvirt* [production]
06:01 <l10nupdate@tin> ResourceLoader cache refresh completed at Wed Oct 7 06:01:01 UTC 2015 (duration 1m 0s) [production]
05:25 <ebernhardson> generating elasticsearch indices in codfw, should run ~3 hours [production]
04:33 <ebernhardson@tin> Synchronized wmf-config/InitialiseSettings.php: Touch InitiialiSettings.php to force config regeneration (duration: 00m 18s) [production]
04:23 <ebernhardson@tin> Synchronized wmf-config/: enable second es cluster in testwiki one more time (duration: 00m 18s) [production]
04:15 <ebernhardson> sync-common on mw2187.codfw.wmnet to fix localisation cache errors in exception.log [production]