4751-4800 of 10000 results (47ms)
2016-09-29 §
16:34 <elukey> executed 'sudo salt -C 'G@cluster:imagescaler and G@site:eqiad' cmd.run 'find /var/log/hhvm/ -type f -user root -exec chown www-data:www-data {} \;' to reduce cronspam [production]
16:33 <elukey> executed 'sudo salt -C 'G@cluster:imagescaler and G@site:codfw' cmd.run 'find /var/log/hhvm/ -type f -user root -exec chown www-data:www-data {} \;' to reduce cronspam [production]
16:32 <urandom> T133395: restbase staging: starting bootstrap of restbase-test2001-b.codfw.wmnet (test of decomm/bootstrap under time-windowed compaction) [production]
15:18 <urandom> T133395: restbase staging: decommissioning restbase-test2001-b.codfw.wmnet (test of decomm/bootstrap under time-windowed compaction) [production]
13:28 <cwd|afk> disabled Adyen job runner [production]
10:29 <hashar> Upgrading Jenkins plugins with zeljkof :] [production]
08:38 <reedy@tin> Synchronized wmf-config/mobile-labs.php: Remove transfers of non existent $wmg variables (duration: 00m 48s) [production]
02:14 <eileen> civicrm upgraded from 86371233a9218a526f85cd76d7008f966042fde7 to 6768613907598c52b0af1ffae35faa9078f15f63 [production]
00:57 <cwd> updated SmashPig from 3458f93599084815da46a2540e9ed762c8b120ce to 3d0a76b5918f280602e5dabeeb0373c1a118590d [production]
2016-09-28 §
23:51 <ejegg> updated SmashPig from d2589271d43d8895fbb928f54f8feb43fcbc43c9 to 3458f93599084815da46a2540e9ed762c8b120ce [production]
22:06 <cwd> PaymentListeners rolled back [production]
21:54 <cwd> PaymentListeners updated from b4d77a991e100f97d98fcd72eaf03940a4e1845d to 21647c8f4b781b74ae2dc4377334410b4eed7 [production]
21:33 <Krenair> Fixed labs 205.21.68.10.in-addr.arpa. entry to remove another broken contintcloud name, unbreaking beta scap [production]
21:25 <XenoRyet> update SmashPig from 372cd4008fee3fd02ad2eae9163cf7b28d2ef7c8 to d2589271d43d8895fbb928f54f8feb43fcbc43c9 [production]
20:51 <apergos> is awesome and made the bot work again by restarting it [production]
20:50 <apergos> restarted logmsgbot on neon [production]
20:47 <bd808> logmsgbot seems to be down: "error: [Errno 111] Connection refused" from scap sync-file [production]
20:46 <bd808> scap sync-file wmf-config/throttle.php "IP cap lift for eswiki on 2016-09-30 (T146788)" [production]
19:48 <awight> update fundraising CRM from 6b2bd98fce5006030423bccc4f4b7fd9b5d14821 to 86371233a9218a526f85cd76d7008f966042fde7 [production]
19:22 <XenoRyet> reverted SmashPig from 4b930ada57d99070713be18e235238078e1c1e48 to 372cd4008fee3fd02ad2eae9163cf7b28d2ef7c8 [production]
17:22 <jynus> restarting db1069.s3 (stagnant replication) [production]
16:42 <awight> update CRM from 88000acc826592e535491bde6a47f8976deaf07a to 6b2bd98fce5006030423bccc4f4b7fd9b5d14821 [production]
08:15 <twentyafterfour> twentyafterfour@iridium:/srv/phab/phabricator$ sudo bin/search index --type PhabricatorProject --force [production]
03:34 <eileen> tools upgrade from b0be0f9ca04191c4bab869bb81191c5c77c432ca to 5427a601ed64ecfe01ef84eda13f28c0dd08a3af [production]
01:20 <eileen> updating CiviCRM from 26e5214e66f2e4b5de89c27dd5121dfde43269e2 to 88000acc826592e535491bde6a47f8976deaf07a [production]
01:05 <eileen> civicrm upgrade from d30a5e454e6614214faef8354a3a23d71d573c7f to 26e5214e66f2e4b5de89c27dd5121dfde43269e2 [production]
2016-09-27 §
05:32 <_joe_> rebooting ms-be1002, stuck in a failed disk [production]
2016-09-26 §
19:37 <awight> enabling awight_test5 banner at 1% of nlwiki [production]
18:09 <ejegg> rolled back paypal IPN listener to b4d77a991e100f97d98fcd72eaf03940a4e1845d [production]
17:59 <ejegg> updated standalone paypal IPN listener from b4d77a991e100f97d98fcd72eaf03940a4e1845d to 21647c8f4b781b74ae2dc4377334410b4eed7e3c [production]
17:47 <ejegg> rolled back paypal IPN listener to b4d77a991e100f97d98fcd72eaf03940a4e1845d [production]
17:39 <ejegg> updated standalone paypal IPN listener from b4d77a991e100f97d98fcd72eaf03940a4e1845d to 21647c8f4b781b74ae2dc4377334410b4eed7e3c [production]
13:52 <marostegui> phabricator is back in write mode - search is degraded. we are regenerating the indexes [production]
13:52 <chasemp> iridium phab ./bin/search index --all [production]
03:39 <cwdent_> disabled civicrm dedupe contacts job [production]
2016-09-25 §
16:25 <Amir1> deploying 9cc2009 on ORES nodes (T146581) [production]
15:24 <elukey> executed https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hadoop/Administration#Fixing_HDFS_mount_at_.2Fmnt.2Fhdfs on stat1002 (fusermount didn't succeed to umount though) [production]
14:39 <gehel> restarting blazegraph on wdqs1002 - T146576 [production]
14:30 <gehel> putting wdqs1002 in maintenance mode, server looks unstable, investigating... - T146576 [production]
14:28 <godog> temporarily redirect ores-celery-worker logs to /srv/log/celery/syslog.log and remove old daemon.log.1 to avoid scb100* filling up the / filesystem [production]
2016-09-24 §
19:30 <ema> hhvm 1283-1290 rolling restart [production]
12:21 <godog> apply temporary cleanup of old (+20m) thumbor temporary files - T146262 [production]
10:47 <_joe_> systemctl restart thumbor-instances.service on thumbor1001 freed 3 GB of space [production]
02:44 <l10nupdate@tin> ResourceLoader cache refresh completed at Sat Sep 24 02:44:59 UTC 2016 (duration 5m 57s) [production]
02:39 <mwdeploy@tin> scap sync-l10n completed (1.28.0-wmf.20) (duration: 16m 49s) [production]
2016-09-23 §
22:05 <matt_flaschen> Deployed patch for T146425 [production]
21:42 <ebernhardson@tin> Synchronized php-1.28.0-wmf.20/extensions/CirrusSearch/includes/ElasticsearchIntermediary.php: Additional logging to track down autocomplete timing regression (duration: 00m 50s) [production]
20:52 <gehel> cleaning up leftover system unit files on wdqs1* [production]
18:41 <gehel> killing stuck tilerator notification processes on maps1001 - T145534 [production]
17:57 <mutante> mira restarted cron [production]