1-50 of 4408 results (5ms)
2009-11-07 §
22:14 <mark> Started puppet on sq44 [production]
22:12 <mark> Unstuck puppet on srv149 [production]
22:02 <mark> Fixed puppet on ixia [production]
21:48 <mark> Installed puppet on yongle [production]
21:26 <mark> Rolling dist-upgrade finished [production]
21:12 <midom> synchronized php-1.5/wmf-config/db.php 'removing db21 from rotation - will be used for migration source' [production]
18:35 <mark> Moved planet.wikimedia.org DNS CNAMEs to singer [production]
18:35 <mark> Setup planet.wikimedia.org instance on singer [production]
17:09 <mark> Moved noc.wikimedia.org CNAME to fenari [production]
17:09 <mark> Setup noc.wikimedia.org instance on fenari, using puppet [production]
14:57 <mark> Doing rolling dist-upgrade of all application servers [production]
14:13 <mark> Removed optsview APT repository on srv225 [production]
14:12 <mark> Removed optsview APT repository on srv224 [production]
13:36 <mark> Manually upgrading puppet across the cluster to help itself get past a catalog bug [production]
13:03 <mark> Setup puppet to automatically upgrade wikimedia-task-appserver on all application servers; in this case it will roll out package php5-geoip across the cluster [production]
09:12 <ariel> synchronized php-1.5/wmf-config/InitialiseSettings.php 'enable Collection on strategy wiki per bug 21361' [production]
2009-11-06 §
23:45 <tfinc> synchronized php-1.5/extensions/ContributionTracking/ContributionTracking_body.php 'dropping 2008 theme' [production]
23:39 <tfinc> synchronized php-1.5/extensions/ContributionReporting/ContributionHistory_body.php 'dropping 2008 theme' [production]
23:19 <mark> Replaced the Webrick puppetmaster setup on sockpuppet by an Apache/Mongrel setup with 4 children to deal with scalability problems [production]
21:29 <mark> Increased active record DB connection pool size from 5 to 50... by just editing the ruby library file [production]
18:54 <mark> db10 replication caught up with db9 [production]
18:08 <mark> Set up puppet to run "apt-get update" on every run (for now, may make that somewhat less aggressive if needed) [production]
17:10 <andrew> synchronized php-1.5/wmf-config/ExtensionMessages.php [production]
17:08 <mark> Setup puppet to exchange all hostkeys between all servers [production]
17:07 <andrew> synchronized php-1.5/includes/LinksUpdate.php [production]
17:05 <Andrew> Scapping [production]
17:05 <Andrew> Set up GlobalUsage on test.wikipedia.org [production]
16:55 <mark> Set up puppet to upgrade itself everywhere [production]
16:40 <mark> Fixed permission problem in the MySQL data dir, restarted replication on db10 [production]
16:37 <mark> Fixed LVM problem on db10 [production]
16:02 <mark> Rebooting db10: OOM errors [production]
15:59 <mark> Replication db9->db10 broken again [production]
15:56 <mark> Set up MySQL database 'puppet' on db9/db10 [production]
2009-11-05 §
19:59 <tfinc> synchronized php-1.5/extensions/DonationInterface/donate_interface/donate_interface.i18n.php [production]
19:51 <tfinc> synchronized php-1.5/extensions/DonationInterface/donate_interface/donate_interface.php [production]
18:48 <mark> Setup puppet Stored Configurations with Stomp queuing; made a custom /usr/local/sbin/puppetqd since it was missing in the puppetmaster package. [production]
17:58 <Rob> transcode1-3 drac setup, servers racked, not in racktables yet, no os yet. [production]
17:42 <mark> Upgraded distribution of sockpuppet to 9.10 (Karmic) [production]
17:35 <Rob> pushing dns changes for mgmt on transcode1-3 [production]
16:54 <mark> Upgraded puppetmaster on sockpuppet to version 0.25.1 [production]
15:35 <mark> Killed all Nagios processes on bart [production]
15:28 <mark> synchronized php-1.5/wmf-config/InitialiseSettings.php 'Disabled InitialiseSettings.php again on all wikis, except testwiki. Pending HTTP proxy setup and security review.' [production]
15:19 <mark> Moved secure.wikimedia.org service IP from bart to singer [production]
15:13 <brion> synchronized php-1.5/wmf-config/InitialiseSettings.php 'enabling upload by URL for sysops on all wikis' [production]
15:09 <mark> Moved NameVirtualHost entries to /etc/apache2/conf.d/namevirtualhost on singer, to stop Apache from complaining [production]
15:06 <mark> Set up secure.wikimedia.org vhost on singer, SSL proxying to the apaches [production]
14:49 <mark> synchronized php-1.5/wmf-config/CommonSettings.php 'Add singer to the SquidServers list' [production]
2009-11-04 §
23:03 <tfinc> synchronized php-1.5/extensions/ContributionTracking/ContributionTracking_body.php [production]
21:09 <mark> Stopped most services on khaldun, to prepare for decommissioning [production]
21:06 <mark> Setup csw5-pmtpa to use brewster as dhcp ip helper [production]