551-600 of 10000 results (40ms)
2016-12-26 §
19:14 <otto@tin> Finished deploy [eventstreams/deploy@90934c3]: (no message) (duration: 00m 31s) [production]
19:13 <otto@tin> Starting deploy [eventstreams/deploy@90934c3]: (no message) [production]
19:13 <otto@tin> Finished deploy [eventstreams/deploy@581a5a1]: (no message) (duration: 00m 22s) [production]
19:13 <otto@tin> Starting deploy [eventstreams/deploy@581a5a1]: (no message) [production]
19:10 <otto@tin> Finished deploy [eventstreams/deploy@836b441]: (no message) (duration: 01m 16s) [production]
19:09 <otto@tin> Starting deploy [eventstreams/deploy@836b441]: (no message) [production]
19:08 <otto@tin> Finished deploy [eventstreams/deploy@e771863]: (no message) (duration: 01m 05s) [production]
19:07 <otto@tin> Starting deploy [eventstreams/deploy@e771863]: (no message) [production]
19:07 <otto@tin> Finished deploy [eventstreams/deploy@e771863]: log (duration: 00m 03s) [production]
19:07 <otto@tin> Starting deploy [eventstreams/deploy@e771863]: log [production]
19:07 <otto@tin> Finished deploy [eventstreams/deploy@e771863]: (no message) (duration: 00m 04s) [production]
19:07 <otto@tin> Starting deploy [eventstreams/deploy@e771863]: (no message) [production]
19:07 <otto@tin> Finished deploy [eventstreams/deploy@e771863]: (no message) (duration: 00m 10s) [production]
19:06 <otto@tin> Starting deploy [eventstreams/deploy@e771863]: (no message) [production]
12:09 <hashar> beta: restarted varnish.service and varnish-frontend.service on deployment-cache-text04 [releng]
2016-12-25 §
08:30 <jynus> restarting dbstore1002 mariadb, alive but unresponsive to queries do to long running queries saturating the service [production]
00:28 <yuvipanda> comment out cron running 'clean' script of avicbot every minute without -once [tools]
00:28 <yuvipanda> force delete all jobs of avicbot [tools]
00:25 <yuvipanda> delete all jobs of avicbot. This is 419 jobs [tools]
00:20 <yuvipanda> kill clean.sh process of avicbot [tools]
2016-12-24 §
09:02 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/329038 [releng]
2016-12-23 §
23:42 <bd808> Updated demo server to include password strength meter [striker]
21:29 <mutante> restarted apache on mw canary servers [production]
19:55 <mutante> cobalt - restarted apache [production]
19:46 <mutante> install libxml2 upgrade on bastions and mw-canary [production]
19:38 <mutante> install libxml2 upgrade on "misc-others" [production]
16:32 <robh> rebooted db2060 due to some kind of hw raid or xfs failure, wouldnt take input [production]
15:13 <jynus> testing labsdb-analytics automatic failover [production]
12:18 <legoktm> deploying https://gerrit.wikimedia.org/r/328886 [releng]
09:23 <jynus> stopping slave dbstore2001(s2) and db2034 for more table cloning [production]
07:56 <moritzm> installing Python security updates [production]
2016-12-22 §
23:16 <ejegg> updated fundraising dashboard from c2cccc058e3e7f929dca5c733c29a3054bca6712 to bec0077599737c774cc716851171365b38a8b02a [production]
22:46 <ejegg> updated fundraising dashboard from ea4305bc9b2c919d3f828ba542eb32b91b207af7 to c2cccc058e3e7f929dca5c733c29a3054bca6712 [production]
22:11 <thcipriani> disable production l10nupdate for deployment freeze [releng]
22:11 <thcipriani> disable l10nupdate cron for deployment freeze [production]
21:25 <ebernhardson> restarting elasticsearch (again) on relforge100[12] to test ltr plugin [production]
21:10 <RoanKattouw> Got lots of errors like 20:49:56 Unable to find remote tracking branch/tag for /srv/mediawiki-staging/php-1.29.0-wmf.6/extensions/ParserFunctions during this scap [production]
21:03 <catrope@tin> Finished scap: Sync Idf4618977f172 in the OAuth extension- (duration: 24m 16s) [production]
20:39 <catrope@tin> Started scap: Sync Idf4618977f172 in the OAuth extension- [production]
20:11 <bd808> Killed running pod with `kubectl delete pod bash-1448508424-kd6kx` and then followed up with `webservice --backend=kubernetes start` [tools.bash]
20:10 <bd808> Service was down, `webservice status` said not running, `kubectl get pods` disagreed [tools.bash]
19:59 <ebernhardson> restarting elasticsearch (again) on relforge100[12] to test ltr plugin [production]
19:22 <gehel> restart wdqs-blazegraph and wdqs-updater on wdqs1001.eqiad.wmnet (suspicious load) [production]
19:18 <ebernhardson> restarting elasticsearch on relforge100[12] to test ltr plugin [production]
19:18 <jynus> stopping replication on dbstore2001(s2) and db2035 for enwiktionary.templatelinks reimport [production]
19:04 <godog> roll restart swift proxy on ms-fe1* to drain thumbor traffic [production]
15:39 <jynus> restart dbstore2001 to change buffer pool size, testing gerrit:328671 [production]
15:28 <elukey> changed firewall rules to allow only $ANALYTICS_NETWORKS (rather than the broader $INTERNAL) for the Yarn UI http service (an1001) and the hive metastore (an1003) [analytics]
14:51 <elukey> restarting the yarn node manager java daemons on all the Hadoop worker nodes due to suspect memory leak [production]
14:14 <elukey> the previous entry is missing: "on analytics1032" [production]