|
2016-09-26
§
|
| 17:59 |
<ejegg> |
updated standalone paypal IPN listener from b4d77a991e100f97d98fcd72eaf03940a4e1845d to 21647c8f4b781b74ae2dc4377334410b4eed7e3c |
[production] |
| 17:47 |
<ejegg> |
rolled back paypal IPN listener to b4d77a991e100f97d98fcd72eaf03940a4e1845d |
[production] |
| 17:39 |
<ejegg> |
updated standalone paypal IPN listener from b4d77a991e100f97d98fcd72eaf03940a4e1845d to 21647c8f4b781b74ae2dc4377334410b4eed7e3c |
[production] |
| 13:52 |
<marostegui> |
phabricator is back in write mode - search is degraded. we are regenerating the indexes |
[production] |
| 13:52 |
<chasemp> |
iridium phab ./bin/search index --all |
[production] |
| 11:35 |
<hashar> |
deployment-salt02 : autoremoving a bunch of java related packages |
[releng] |
| 11:31 |
<hashar> |
rebooting deployment-salt02 has a kernel soft lock while hitting the disk |
[releng] |
| 11:24 |
<hashar> |
beta: mass upgrading all debian packages on all instances |
[releng] |
| 10:32 |
<hashar> |
beta: on deployment-pdf01 rm -fR /home/cscott/tmp/npm* |
[releng] |
| 10:29 |
<hashar> |
deployment-pdf01 apt-get upgrade / cleaning files left over etc |
[releng] |
| 10:28 |
<hashar> |
beta: on deployment-pdf01 rm -fR /home/cscott/.npm/ T145343 |
[releng] |
| 03:39 |
<cwdent_> |
disabled civicrm dedupe contacts job |
[production] |
|
2016-09-25
§
|
| 16:25 |
<Amir1> |
deploying 9cc2009 on ORES nodes (T146581) |
[production] |
| 16:03 |
<Amir1> |
deploying c2ba205 (T146581) |
[ores] |
| 15:59 |
<Amir1> |
c2ba205 going to staging |
[ores] |
| 15:27 |
<Krenair> |
restarted labs-logbot under tools.morebots |
[tools] |
| 15:26 |
<halfak> |
restarted precached and uwsgi-ores on ores-web-03 |
[ores] |
| 15:24 |
<elukey> |
executed https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hadoop/Administration#Fixing_HDFS_mount_at_.2Fmnt.2Fhdfs on stat1002 (fusermount didn't succeed to umount though) |
[production] |
| 15:23 |
<halfak> |
restarted precached and uwsgi-ores on ores-web-03 |
[ores] |
| 14:39 |
<gehel> |
restarting blazegraph on wdqs1002 - T146576 |
[production] |
| 14:30 |
<gehel> |
putting wdqs1002 in maintenance mode, server looks unstable, investigating... - T146576 |
[production] |
| 14:28 |
<godog> |
temporarily redirect ores-celery-worker logs to /srv/log/celery/syslog.log and remove old daemon.log.1 to avoid scb100* filling up the / filesystem |
[production] |
|
2016-09-23
§
|
| 22:05 |
<matt_flaschen> |
Deployed patch for T146425 |
[production] |
| 21:42 |
<ebernhardson@tin> |
Synchronized php-1.28.0-wmf.20/extensions/CirrusSearch/includes/ElasticsearchIntermediary.php: Additional logging to track down autocomplete timing regression (duration: 00m 50s) |
[production] |
| 20:52 |
<gehel> |
cleaning up leftover system unit files on wdqs1* |
[production] |
| 20:49 |
<halfak> |
restarted ores-uwsgi and precached services on ores-web-03 |
[ores] |
| 20:41 |
<yuvipanda> |
add halfak as projectadmin |
[quarry] |
| 20:21 |
<hashar> |
integration: salt -v '*trusty*' cmd.run 'service mysql start' |
[releng] |
| 20:00 |
<hashar> |
rebooting all CI permanent slaves. Making sure nothing is left on /mnt (which is no more mounted) |
[releng] |
| 19:53 |
<hashar> |
added a 30 minutes build timeout to https://integration.wikimedia.org/ci/job/phabricator-jessie-diffs/ |
[releng] |
| 18:41 |
<gehel> |
killing stuck tilerator notification processes on maps1001 - T145534 |
[production] |
| 17:57 |
<mutante> |
mira restarted cron |
[production] |
| 17:53 |
<ejegg> |
updated SmashPig from 8ac116037440746eaf64b9e99e1ee962d5d33475 to 372cd4008fee3fd02ad2eae9163cf7b28d2ef7c8 |
[production] |
| 17:46 |
<thcipriani@tin> |
Synchronized README: Test sync for new mira (duration: 01m 27s) |
[production] |
| 17:43 |
<mutante> |
mira - changing UID of l10nupdate to 10002, chown'ing files (1001 -> 10002) |
[production] |
| 17:35 |
<ebernhardson@tin> |
Synchronized php-1.28.0-wmf.20/extensions/CirrusSearch/includes/ElasticsearchIntermediary.php: Add timing marks to narrow down autocomplete timing regression (duration: 00m 50s) |
[production] |
| 17:31 |
<ebernhardson@tin> |
Synchronized php-1.28.0-wmf.20/extensions/CirrusSearch/includes/CompletionSuggester.php: Add timing marks to narrow down autocomplete timing regression (duration: 18m 43s) |
[production] |
| 17:04 |
<mutante> |
stat1002 - before it was hanging and then fixed due to https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hadoop/Administration#Fixing_HDFS_mount_at_.2Fmnt.2Fhdfs |
[production] |
| 17:03 |
<mutante> |
stat1002 - starting nagios-nrpe-server |
[production] |
| 15:02 |
<hashar> |
rebooting integration-slave-jessie-1001 |
[releng] |
| 14:55 |
<jynus> |
deployed dns update (removing db1010) T129395 |
[production] |
| 14:04 |
<hashar> |
remove the /mnt based tmpfs for T146381 / https://gerrit.wikimedia.org/r/#/c/312518/ via: salt -v '*' cmd.run 'umount /mnt/home/jenkins-deploy/tmpfs' |
[releng] |
| 13:41 |
<hashar> |
Switching tmpfs from /mnt to /srv https://gerrit.wikimedia.org/r/#/c/312330/ and running fab deploy_slave_scripts |
[releng] |
| 12:20 |
<moritzm> |
rearmed keyholder on mira |
[production] |