|
2011-11-18
§
|
| 16:59 |
<binasher> |
updated gmetad on spence to use db21 as a data source, quieting cron spam |
[production] |
| 11:11 |
<apergos> |
that message is from much earlier but I forgot to hit "enter" somehow... |
[production] |
| 11:10 |
<apergos> |
changed smokeping config on streber to check db40 instead of db20 in rack b2 sdtpa |
[production] |
| 02:20 |
<LocalisationUpdate> |
completed (1.18) at Fri Nov 18 02:23:11 UTC 2011 |
[production] |
| 01:57 |
<LocalisationUpdate> |
failed |
[production] |
| 01:01 |
<binasher> |
packaged graphite, added to wikimedia-lucid, installed on professor via puppet |
[production] |
| 00:43 |
<neilk> |
synchronized php-1.18/extensions/MoodBar/FeedbackItem.php 'fix for Moodbar code which apparently cannot deal with empty comment records' |
[production] |
| 00:21 |
<neilk> |
rebuilt wikiversions.cdb and synchronized wikiversions files: |
[production] |
| 00:20 |
<neilk> |
synchronizing Wikimedia installation... : Moodbar phase 1 bugfixes |
[production] |
|
2011-11-17
§
|
| 23:43 |
<K4-713> |
Synchronized payments cluster to [[rev:103513|r103513]] |
[production] |
| 23:25 |
<notpeter> |
upgrading and restarting udp2log on emery |
[production] |
| 22:08 |
<binasher> |
added db20 to decommissioning.pp, renamed server (professor) will be the new mediawiki profiling host |
[production] |
| 20:23 |
<RobH> |
torrus deadlock, restarting on streber |
[production] |
| 18:27 |
<cmjohnson1> |
virt13 power inadvertently removed |
[production] |
| 17:23 |
<cmjohnson1> |
db45 power was removed during new install |
[production] |
| 09:27 |
<mutante> |
started ActiveMQ on erzurumi (RT#1999) |
[production] |
| 07:43 |
<awjr> |
activemq disabled on erzurumi |
[production] |
| 07:05 |
<Tim> |
on fenari: added wmerrors UDP log receiver to /etc/rc.local as a temporary solution pending RT 623 |
[production] |
| 06:48 |
<Tim> |
put the new version of wmerrors into the apt repo |
[production] |
| 06:23 |
<awjr> |
redisabled donations queue consumer in jenkins on aluminium |
[production] |
| 06:16 |
<awjr> |
reenabled donations queue consumer in jenkins on aluminium |
[production] |
| 05:57 |
<awjr> |
disabling donations queue consumer in jenkins on aluminium |
[production] |
| 02:25 |
<LocalisationUpdate> |
completed (1.18) at Thu Nov 17 02:28:16 UTC 2011 |
[production] |
|
2011-11-16
§
|
| 21:45 |
<cmjohnson1> |
sq86 shutdown |
[production] |
| 21:16 |
<cmjohnson1> |
sq85 shutdown |
[production] |
| 20:55 |
<cmjohnson1> |
shutdown sq84 |
[production] |
| 20:26 |
<cmjohnson1> |
shutdown sq83 |
[production] |
| 19:48 |
<Ryan_Lane> |
shut down sq82 |
[production] |
| 19:39 |
<maplebed> |
grew xfs on ms5 by 2T to get some breathing room |
[production] |
| 19:30 |
<cmjohnson1> |
sq81 down for rack relocation |
[production] |
| 18:19 |
<awjr> |
Increasing donatinos queue consumption script frequency to run every 2 mins rather than 3 mins via Jenkins on aluminium |
[production] |
| 16:26 |
<mutante> |
fixed permissions on nagios external command file (RT #1994) |
[production] |
| 16:12 |
<RobH> |
dns updated for db48-db58 (both mgmt and production) |
[production] |
| 15:39 |
<mutante> |
srv180 - powered down, was offline with disk fail anyways, created RT 1993 |
[production] |
| 13:36 |
<mutante> |
argon - dpkg errors processing mysql-client pkg upgrade |
[production] |
| 13:33 |
<mutante> |
argon - dist-upgrade/kernel,reboot (unused limesurvey box) |
[production] |
| 04:12 |
<awjr> |
synchronizing payments cluster, [[rev:103293|r103293]] |
[production] |
| 03:18 |
<asher> |
synchronized wmf-config/db.php 'adding db19 to s2, running new mysql build' |
[production] |
| 02:24 |
<LocalisationUpdate> |
completed (1.18) at Wed Nov 16 02:27:02 UTC 2011 |
[production] |
| 01:57 |
<LocalisationUpdate> |
failed |
[production] |
| 01:48 |
<^demon> |
leaving a git svn test running in a screen on formey. Should be fine, but feel free to kill it if svn.wm.o explodes. |
[production] |
| 00:43 |
<asher> |
synchronized wmf-config/db.php 'putting db11 (s3) into rotation with new mysql build' |
[production] |
| 00:27 |
<Tim> |
on srv255: ran manual sync-common and puppetd -tv |
[production] |
| 00:19 |
<tstarling> |
rebuilt wikiversions.cdb and synchronized wikiversions files: |
[production] |
| 00:18 |
<Tim> |
rebooting srv255, has been in swapdeath for a week, no response on serial console |
[production] |
| 00:12 |
<tstarling> |
synchronizing Wikimedia installation... : |
[production] |
| 00:05 |
<tstarling> |
synchronized php-1.18/README 'sync test' |
[production] |
| 00:01 |
<binasher> |
rebooting db11 to new kernel (it's out of rotation) |
[production] |