|
2010-09-21
§
|
| 15:21 |
<mark> |
restarted backend squid on sq31 |
[production] |
| 15:16 |
<mark> |
Lowered cache_mem from 2500 to 2000 on sq31-33 |
[production] |
| 09:35 |
<Tim> |
on grosley: tried to fix broken hudson-labs apt source, apt-get dist-upgrade |
[production] |
| 09:31 |
<Tim> |
apt-get dist-upgrade on sanger, bayes |
[production] |
| 09:19 |
<Tim> |
on mchenry: apt-get dist-upgrade, and chmod 755 /var/spool/exim4 so that NRPE works again |
[production] |
| 09:03 |
<Tim> |
configured kaulen to get its packages from the local mirror, and removed the o=Wikimedia pin to make it not use the PHP packages which were compiled on hardy and which conflict with everything |
[production] |
| 08:28 |
<Tim> |
apt-get dist-upgrade on brewster, ersch, erzurumi, formey |
[production] |
| 08:22 |
<Tim> |
on alsted: apt-get dist-upgrade |
[production] |
| 07:13 |
<Tim> |
on hume: apt-get dist-upgrade |
[production] |
| 05:05 |
<tstarling> |
synchronized php-1.5/wmf-config/secure.php 'removed downtime notice' |
[production] |
| 04:58 |
<Tim> |
on yongle: fixed broken sources.list, upgraded 187 packages |
[production] |
| 04:54 |
<Tim> |
upgraded packages on tarin, tridge, williams |
[production] |
| 04:42 |
<Tim> |
on streber: removed python-pysnmp2, conflicted with python-pysnmp4 and apparently wasn't being used |
[production] |
| 04:28 |
<Tim> |
upgraded packages on stafford, streber. python-pysnmp-common on streber is broken |
[production] |
| 04:14 |
<Tim> |
upgraded packages on singer |
[production] |
| 04:07 |
<Tim> |
upgrading packages on spence |
[production] |
| 04:03 |
<Tim> |
upgrading packages on sockpuppet |
[production] |
| 03:53 |
<laner> |
synchronized php-1.5/wmf-config/InitialiseSettings.php |
[production] |
| 03:53 |
<Tim> |
scheduled reboot of singer at 05:00 using atd |
[production] |
| 03:50 |
<tstarling> |
synchronized php-1.5/wmf-config/secure.php 'secure.wikimedia.org scheduled reboot notice' |
[production] |
| 03:46 |
<laner> |
synchronizing Wikimedia installation... Revision: 72920 |
[production] |
| 03:22 |
<Ryan_Lane> |
updated the kernel on oldusability.wikimedia.org linode server, and rebooted |
[production] |
| 02:55 |
<Tim> |
upgrading kernel on wikitech linode to latest 2.6.32, will reboot |
[production] |
| 02:45 |
<laner> |
synchronized php-1.5/extensions/CentralNotice/CentralNotice.php 'removing debug info' |
[production] |
| 02:44 |
<laner> |
synchronized php-1.5/extensions/CentralNotice/SpecialBannerListLoader.php 'removing debug info' |
[production] |
| 02:43 |
<laner> |
synchronized php-1.5/extensions/CentralNotice/SpecialCentralNotice.php 'removing debug info' |
[production] |
| 02:39 |
<tfinc> |
synchronized php-1.5/extensions/CentralNotice/SpecialCentralNotice.php 'removing debug info' |
[production] |
| 02:13 |
<tfinc> |
synchronizing Wikimedia installation... Revision: 72920 |
[production] |
| 02:12 |
<Tim> |
rebooting locke and sanger for kernel upgrade |
[production] |
| 02:09 |
<Tim> |
rebooting hume for kernel upgrade |
[production] |
| 01:33 |
<tfinc> |
ran sync-common-all |
[production] |
| 01:12 |
<Tim-away> |
fixed syntax error in InitialiseSettings.php, which caused the site to go down |
[production] |
|
2010-09-20
§
|
| 22:23 |
<Ryan_Lane> |
finished kernel upgrades on tesla VMs |
[production] |
| 20:54 |
<Ryan_Lane> |
moved mysql data from /var/lib/mysql to /a/sqldata on prototype |
[production] |
| 18:33 |
<Ryan_Lane> |
Correction: previous ufw logs were for eiximenis |
[production] |
| 18:33 |
<Ryan_Lane> |
added ufw application configuration for etherpad. Added allow rules for OpenSSH and Etherpad in ufw config. Enabled ufw. |
[production] |
| 18:32 |
<Ryan_Lane> |
added ufw prerouting rules for 80->9000 for etherpad. |
[production] |
| 18:32 |
<Ryan_Lane> |
restarted etherpad on eiximenis |
[production] |
| 17:46 |
<Ryan_Lane> |
upgrading kernel on all tesla VMs |
[production] |
| 16:30 |
<Ryan_Lane> |
running database recovery on torrus on streber, and restarting the service |
[production] |
| 14:29 |
<Tim> |
rebooting mchenry for kernel upgrade |
[production] |
| 13:39 |
<Tim> |
rebooting ersch, erzurumi, formey and kaulen for new kernel |
[production] |
| 12:34 |
<Tim> |
rebooting brewster and eiximenis for new kernel |
[production] |
| 12:25 |
<Tim> |
updated browne /etc/motd |
[production] |
| 08:49 |
<Tim> |
rebooting alsted for new kernel |
[production] |
| 07:35 |
<Tim> |
rebooting fenari to get new kernel |
[production] |
| 07:03 |
<Tim> |
upgrading everything on fenari |
[production] |