|
2011-06-12
§
|
| 19:47 |
<JeLuF> |
Software RAID on fenari was broken, mirror sdb1 was failed. Re-added sdb1. If the error is persistent, sdb needs to be replaced. |
[production] |
| 19:19 |
<JeLuF> |
cron.{hourly,daily,...} on hume was not running due to a broken record in /etc/crontab, so logrotate was not running |
[production] |
| 12:58 |
<ariel> |
synchronized php-1.17/includes/HistoryBlob.php 'check for existence of mhash function, not extension (need for lucid php build) (take 2, after svn up :-P)' |
[production] |
| 12:51 |
<ariel> |
synchronized php-1.17/includes/HistoryBlob.php 'check for existence of mhash function, not extension (need for lucid php build)' |
[production] |
|
2011-06-09
§
|
| 23:54 |
<binasher> |
upgrading mobile2 to maverick (pre natty) |
[production] |
| 23:04 |
<Ryan_Lane> |
added preilly account as a restricted user. Added him to mobile1-5 |
[production] |
| 22:30 |
<binasher> |
upgrading mobile3 to natty ice |
[production] |
| 21:38 |
<binasher> |
upgrading mobile3 to maverick (then natty) |
[production] |
| 20:02 |
<catrope> |
synchronizing Wikimedia installation... Revision: 89789: |
[production] |
| 20:02 |
<RoanKattouw> |
Running scap to deploy ArticleFeedback changes |
[production] |
| 18:59 |
<binasher> |
mobile5 natty |
[production] |
| 18:42 |
<binasher> |
mobile5 upgrade to maverick on the way to natty |
[production] |
| 15:28 |
<catrope> |
synchronized php-1.17/wmf-config/CommonSettings.php 'Raise account creation throttle for Lennart's IP on testwiki' |
[production] |
| 15:10 |
<RobH> |
dns updated, all nameservers dig correctly. |
[production] |
| 15:10 |
<RobH> |
pushing dns updates to add mgmt of all eqiad devices |
[production] |
| 01:22 |
<Ryan_Lane> |
shutting off services on virt1 (including puppet) until I get a chance to firewall them properly |
[production] |
|
2011-06-08
§
|
| 22:35 |
<Ryan_Lane> |
rebooting virt1 |
[production] |
| 21:58 |
<hcatlin> |
deploying to mobile cluster |
[production] |
| 21:35 |
<Ryan_Lane> |
changing controller.labs.wikimedia.org to a CNAME for virt1.wikimedia.org |
[production] |
| 21:27 |
<mark> |
Started ircd on ekrem |
[production] |
| 21:20 |
<mark> |
Moved routing of vlan 101 from csw1-sdtpa to csw5-pmtpa |
[production] |
| 21:06 |
<binasher> |
changing pybal healthcheck url for mobile |
[production] |
| 20:47 |
<Ryan_Lane> |
rebooting controller.labs.wikimedia.org |
[production] |
| 20:44 |
<Ryan_Lane> |
restarting pdns on ns0, it's locked up |
[production] |
| 20:33 |
<Ryan_Lane> |
make that deleting virt1.pmtpa.wmnet, and changing virt1.mgmt.pmtpa.wmnet to controller.labs.mgmt.pmtpa.wmnet |
[production] |
| 20:32 |
<Ryan_Lane> |
changing virt1.pmtpa.wmnet to controller.labs.pmtpa.wmnet |
[production] |
| 20:27 |
<Ryan_Lane> |
changing ip of controller.labs.wikimedia.org |
[production] |
| 20:17 |
<mark> |
Rebooting ekrem |
[production] |
| 20:16 |
<mark> |
Changed ekrem's ip |
[production] |
| 20:10 |
<mark> |
Added 2nd ip to ekrem (208.80.152.178) |
[production] |
| 19:56 |
<binasher> |
upgrade mobile4 to natty |
[production] |
| 19:53 |
<mark> |
Shutdown ekrem's switch port and null routed the ip for a while |
[production] |
| 19:53 |
<Ryan_Lane> |
deleting vmdev1 and vmdev2, and moving the controller.labs.wikimedia.org entry into the public services subnet |
[production] |
| 19:45 |
<binasher> |
upgrading mobile4 to maverick on its way to natty |
[production] |
| 19:31 |
<Ryan_Lane> |
reclaiming sandbox ips for owa.tesla, owa1.tesla, owa2.tesla, lvs1.tesla, and grid.tesla |
[production] |
| 18:10 |
<Ryan_Lane> |
depooling mobile5 |
[production] |
| 18:05 |
<Ryan_Lane> |
changing weight for mobile5 |
[production] |
| 17:42 |
<Ryan_Lane> |
upping the weight of mobile5 to 24 |
[production] |
| 17:27 |
<Ryan_Lane> |
upping weight of mobile5 to 16 |
[production] |
| 17:18 |
<Ryan_Lane> |
depooling mobile4 |
[production] |
| 17:16 |
<Ryan_Lane> |
reducing weight of mobile4/5 to 8 |
[production] |
| 17:13 |
<Ryan_Lane> |
repooling mobile4/5 with a weight of 16 |
[production] |