Great for small teams but the issue is that wikis inherently encourage large team collaboration and so small teams ultimately grow to larger ones, and larger teams means larger storage sets and database sizes and when that happens event the best laid plans can turn into weeks of troubleshooting and hair pulling to try to make Apples migration techniques work.

So lets say thats where you are you have a wiki system you are trying to move, you used the method outlined here and no dice. What do you do? Luckily starting in 10.7 Apple moved away from plist storage for their wiki and started using a PostgreSQL database. This is good news, because this means that its in a not easily corruptible format and is easy to extract. Not only that but its nearly tamperproof so accidentally deleting it is much harder than you think.

So how do I move my wiki, glad you asked. I recently helped out with a migration and I learned a few things in the process that surprised even me. The wiki migration steps outlined by Apple are 80% accurate. The method is to move the file storage (images, and attachments) and then export the database as a database dump.

Here is what worked for me.

On the source OS X server, perform this command in Terminal as an administrator in order to dump the Postgres database to a file:

sudo pg_dump --format=c --compress=9 --blobs --username=collab --file=/tmp/collab.pgdump collab

Copy /tmp/collab.pgdump from the source server to /tmp/collab.pgdump on the destination server, then copy the contents of /Library/Server/Wiki/FileData on the source server to /Library/Server/Wiki/FileData on the destination server.

Log in to the destination server as an administrator and execute the following commands in Terminal to ensure correct ownership and permissions, start the Postgres database, populate it with the data dumped from the source server, and finally start up the wiki service:

sudo chown -R _teamsserver:_teamsserver /Library/Server/Wiki/FileData

sudo chmod -R +a "www allow read" /Library/Server/Wiki/FileData

sudo serveradmin stop wiki

sudo serveradmin start postgres

sudo rake -f /usr/share/collabd/server/Rakefile db:drop

sudo createuser -U _postgres -d -s collab

sudo createdb -U collab collab

Ok so far its pretty much the same well here is where it gets interesting. The database export on Apples page does not work well with large database sets. So to compensate you can dump the contents to a .sql file instead a dump file and get better results.

cd /usr/bin

./pg_dump -U _postgres collab -c -f /Library/Server/PostgreSQL/Backup/collab.sql

This exports the data, once done copy that sql file to the destination server or OS. To restore the sql file to another system do the following.

psql -U _postgres -d collab -f /Path/To/The/SQL/File/collab.sql

sudo serveradmin start wiki

If you follow these steps in the order I have written them in, you should be good to go. There is just one major Gotcha. Lets say your moving to a new server and you are thinking of rebuilding your Open Directory. Keep in mind that the entire wiki system hard codes each article, user account, group account, images and more off of the GUID in the OD account.

This means that if you want your wiki to function you need to migrate over the OD in the Apple recommended way, or Export your Users and Groups to retain the GUID information. there is no way around this unfortunately. I hope this helps, alternatively if you are interested in a better wiki solution I have worked with the following free alternatives.

BuddyPress, Open Atrium, and Media Wiki are all good ones, for a good paid Wiki check out Confluence.

WinPE
So what is WinPE? WinPE is a no frills lightweight version of Windows that you can apply to a USB/DVD/CD device to boot from. It can be customized with specific applications and diagnostic tools. To start you need to download the Windows 7 AIK which will allow us to use the most recent version of WinPE which at the time of this article is 3.0. For the sake of this tutorial I should mention that I am running Windows 7 on my OSX computer using VMWare Fusion.

Install the AIK on your computer by burning the .iso file that you downloaded to a DVD or use a program like PowerISO to mount the ISO on the Windows computer from there you can run the setup wizard. Note that you must mount the .iso as a disk in the DVD / CD drive in order for you to install it or you need to burn it and then install from that disk.

Boot Disk
We are going to create a bootable USB drive. Insert a USB drive in your computer, note that we are going to format this drive to be a bootable Windows drive so be sure you can use it fully for this purpose. Once its inserted make sure its visible in the Windows environment. Launch the cmd prompt on the Windows computer and make sure that you have started it up with elevated permissions.

We are going to use the DISKPART command here, this command will allow us to format the drive, clean it, partition it and format it for use with WinPE.

C:Windowssystem32>DISKPART

DISKPART> LIST DISK

  DISK ###  STATUS          SIZE       FREE       DYN     GPT
  --------  --------------- ---------- ---------- ------- --------
  DISK 0    ONLINE          60GB       0B
  DISK 6    ONLINE          14GB       0B  

DISKPART> SELECT DISK {ENTER YOUR DISK NUMBER HERE}

DISKPART> LIST DISK

  DISK ###  STATUS          SIZE       FREE       DYN     GPT
  --------  --------------- ---------- ---------- ------- --------
  DISK 0    ONLINE          60GB       0B
* DISK 6    ONLINE          14GB       0B  

DISKPART> CLEAN

  DISKPART SUCCEEDED IN CLEANING THE DISK

DISKPART> CREATE PARTITION PRIMARY

  DISKPART SUCCEEDED IN CREATING THE SPECIFIED PARTITION

DISKPART> FORMAT FS=FAT32 QUICK

  100 PERCENT COMPLETED
  DISKPART SUCCESSFULLY FORMATTED THE VOLUME.

DISKPART> ACTIVE

  DISKPART MARKED THE CURRENT PARTITION AS ACTIVE.

DISKPART> EXIT

Lets talk briefly about what it is exactly that we see here. We are using Diskpart a program that partitions, cleans and formats external and internal media. We launch the program, and list the active disks that we can use. We select the disk that we want to format and partition. In this example for me that was Disk 1 after I selected this disk you can see that it has an * symbol next to the title. This lets me know that I have selected the right disk. Once done I clean, partition and format the drive. Once that is done I set the drive to Active which is what lets Windows know that this is now a bootable drive.

Create the WinPE Environment
After you download the AIK on your computer you can now use the tools that have come with it. We are going to launch the shortcut in the Microsoft Windows AIK under the Start menu called “Deployment Tools Command Prompt” you want to make sure you launch this with elevated permissions. Once done this is what you should see.

Updating path to include dism, oscdimg, imagex

 C:Program FilesWindows AIKToolsPETools
 C:Program FilesWindows AIKToolsPETools..AMD64;C:Program FilesWIndows AIKToolsPETools..x86
 C:Program FilesWindows AIKToolsPETools..AMD64Servicing;C:Program FilesWindows AIKToolsPETools..x86Servicing;

 C:Program FilesWindows AIKToolsPETools>

So what does this all mean, well it means that you can now run the disk, oscdimg, and imagex programs from this prompt, all of those programs are now available to you. What is oscdimg? Its the utility that converts a folder into an .iso file. What is dism? Dism is the Deployment Imaging Servicing and Management Tool it allows you to manage your .wim files and create custom answer scripts. Finally ImageX is the utility that we will use to capture and manage our windows images.

C:Program FilesWindows AIKToolsPETools>copype
Usage: copyepe [x86 : amd64 : ia64] destination

Example: copyepe x86 c:windowspe-x86

C:Program FilesWindows AIKToolsPETools>copyepe x86 C:windowspe-x86

6 File(s) copied
  1 File(s) copied.

Success

Updating path to include peimg, cdimage, images

 C:Program FilesWindows AIKToolsPETools
 C:Program FilesWindows AIKToolsPETools..AMD64

Now we have the winPE environment copied to the C:windowspe-x86 folder. In this folder you will see the winpe.wim file at the root level. We need to copy this file to the C:windowspe-x86ISOSources folder and rename it to boot.wim. This will define it as the boot image. On top of that we want to be able to use ImageX while we are in the winPE environment so we will also copy that.

C:Program FilesWindows AIKToolsPETools>copy "C:Program FilesWindows AIKtoolsx86imagex.exe" C:windowspe-x86ISO
     1 file(s) copied.

C:Program FilesWindows AIKToolsPETools>copy C:windowspe-x86winpe.wim C:windowspe-x86ISOsourcesboot.wim
     1 file(s) copied.

C:Program FilesWindows AIKToolsPETools>oscdimg -n -bC:windowspe-x86etfsboot.com C:windowspe-x86ISO C:windowspe-x86winpe-imagex.iso

OSCDIMG 2.55 CD-ROM and DVD-ROM Premastering Utility
Copyright (C) Microsoft, 1993-2007. All rights reserved.
Licensed only for producing Microsoft authorized content.


Scanning source tree
Scanning source tree complete (18 files in 8 directories)

Computing directory information complete

Image file is 142673920 bytes

Writing 18 files in 8 directories to C:windowspe-x86winpe-imagex.iso

100% complete

Final image file is 142673920 bytes

Done.

The final command that we ran there was to make the ISO folder into a bootable .iso file that can be copied to the usb drive that we just prepared or burned to a DVD / CD. Now that the ISO is created I am going to mount that file to my E: drive using PowerISO and then copy the contents to my USB Drive.

C:Program FilesWindows AIKToolsPETools>xcopy E:* F: /k /e /d
E:BOOTBCD
E:BOOTBOOT.SDI
E:BOOTBOOTFIX.BIN
E:BOOTETFSBOOT.COM
E:BOOTFONTSCHS_BOOT.TTF
E:BOOTFONTSCHT_BOOT.TTF
E:BOOTFONTSJPN_BOOT.TTF
E:BOOTFONTSKOR_BOOT.TTF
E:BOOTFONTSWGL4_BOOT.TTF
E:EFIMICROSOFTBOOTBCD
E:EFIMICROSOFTBOOTFONTSCHS_BOOT.TTF
E:EFIMICROSOFTBOOTFONTSCHT_BOOT.TTF
E:EFIMICROSOFTBOOTFONTSJPN_BOOT.TTF
E:EFIMICROSOFTBOOTFONTSKOR_BOOT.TTF
E:EFIMICROSOFTBOOTFONTSWGL4_BOOT.TTF
E:SOURCESBOOT.WIM
16 File(s) copied

C:Program FilesWindows AIKToolsPETools>

Once the copy is complete we can now startup the computer using your winPE key. To do this boot up and press the F12 key at startup and choose boot from USB Device. Once done you will boot into the winPE environment that you created. The OS will load from the boot.wim file and you will be able to use ImageX. For the next step we will create a Windows Image from a reference computer and then apply that image to a blank drive.

Reference Computer
For Mac OSX administrators the concept of a reference computer is a little foreign. Where in the Mac World we would use System Image Utility or InstaDMG to create our vanilla system images, Windows users use a reference computer and make sure that they sysprep the drive for the OOBE or Out of the box experience. The concept here is that you will install Windows 7 on a machine and switch to Audit Mode (Shift + Control + F3). Once in Audit mode you can install applications, drivers, set preferences and more. Once done here you need to run sysprep you can run that from the command line or from the GUI interface. Here is what the command looks like in the CMD window.

C:WindowsSystem32sysprepsysprep.exe /oobe /generalize /shutdown

This command will remove all of the user specific files and restore files. Generalize will reset the Windows Activation (Which can only be done 3 times). OOBE will start the user setup process on launch and Shutdown will turn off the computer.

Now that the reference computer is ready we will capture an image based off of that drive. We will need a place to store this image. I will store this on an external hard drive.

Capturing a Windows Image
Now that I have my bootable USB I am going to boot off of it in VMWare Fusion by choosing the wine-imagex.iso that I created. Once done boot the computer and make sure you boot from the disk. You can also boot off of the USB drive you created by pressing F12 at startup and choosing “USB Device” as a boot option.

Once you have booted into winPE you must connect your external hard drive. Once done run Diskpart now and list the volumes. Once done you want to make sure your drive is in the list if not you have to assign it a drive letter.

DISKPART> list volume

  VOLUME ###  LTR  LABEL   FS    TYPE        SIZE   STATUS   INFO
  --------    ---  -----   ----  ----------- ------ -------  --------
  VOLUME 0    D    CD_ROM  CDFS  DVDROM      139MB  HEALTHY
  VOLUME 1    C            NTFS  PARTITION   139MB  HEALTHY
  VOLUME 2                 FAT32 REMOVABLE   139MB  HEALTHY

DISKPART> select volume 2
DISKPART> list volume

  VOLUME ###  LTR  LABEL   FS    TYPE        SIZE   STATUS   INFO
  --------    ---  -----   ----  ----------- ------ -------  --------
  VOLUME 0    D    CD_ROM  CDFS  DVDROM      139MB  HEALTHY
  VOLUME 1    C            NTFS  PARTITION   60GB   HEALTHY
* VOLUME 2                 FAT32 REMOVABLE   14GB   HEALTHY

DISKPART> assign letter=g
DISKPART> list volume

  VOLUME ###  LTR  LABEL   FS    TYPE        SIZE   STATUS   INFO
  --------    ---  -----   ----  ----------- ------ -------  --------
  VOLUME 0    D    CD_ROM  CDFS  DVDROM      139MB  HEALTHY
  VOLUME 1    C            NTFS  PARTITION   60GB   HEALTHY
* VOLUME 2    G            FAT32 REMOVABLE   14GB   HEALTHY

DISKPART> exit

Once you have the drive mounted you can capture the wim image from the reference computer which in this example is Volume 1 or drive C:. As stated before the D: drive is our winPE ISO file that we created. The ImageX program is on this drive so to create the image we run this command.

X:Windowssystem32>d:imagex /capture c: g:win7lab.wim "Windows 7 Image" /compress fast /verify

this will run ImageX from the D: drive capture the C: drive as a windows image, give it a description “Windows 7 Image” and then compress and verify the image. You will notice that it will exclude a lot of files by default and not include them. This is similar to what InstaDMG does on the Mac side. Once done your wim file will be saved to your external disk.

Applying your Windows Image
Once you have booted up off of your winPE device (.iso or USB) on the computer that you want to image you need to run Diskpart again on the blank drive that will get the image.

X:Windowssystem32>diskpart
DISKPART>select disk 0 // SELECT THE DISK THAT WILL GET THE IMAGE
DISKPART>clean
DISKPART>create partition primary
DISKPART>select partition 1
DISKPART>active
DISKPART>format
DISKPART>exit

Executing the format command will take a while depending on the size of your drive as it does a full format rather than a quick format. After the format finishes, and you exit diskpart, you’re ready to actually apply the image. From the WinPE command line, execute the following Imagex command to apply your image

X:Windowssystem32>D:ToolsImagex.exe /apply Y:image.wim 1 C:

The /apply flag tells Imagex to apply the image to the disk. Y:image.wim is the path to the image you wish to apply, in this case I’m assuming you’ve mapped a drive to Y:. The number 1 in the command tells Imagex to apply the first image in the WIM file; chances are you’re only ever going to use 1, but it’s required. Finally, C: is the drive you want to apply the image to.

After executing the Imagex command, you’ll be shown a continually updating progress message. When the process is finished, remove your WinPE CD/USB/.iso and reboot the computer. On first boot from the imaged drive, a new SID will be generated and drivers will be applied. It may take a few minutes and a couple of reboots, but you’ll be playing with your new Windows 7 box in no time.

Lets talk about some of those major flaws. Google Drive and Dropbox are notorious for their sync agents just quitting or exiting quietly in the background. This can be aggravating when in an area with no internet access, you thought you got that last sync in right, nope wrong. How about the fact that you actually have to manually sync, or the lack of the ability to setup a schedule sync in the future. Finally the dreaded message “19 of your 298 files can not be synced”, but why, oh why? These issues can cause even an expert to start pulling out their hair.

So here is the ideal situation, taking your cloud based storage and putting your home folder in the cloud. This will allow you to sync multiple computers together and keep them in sync. For example you have a text file on your desktop, you edit it and it magically shows up on your laptop’s desktop. iCloud… please, this can be done very simply and with a few scripts we can schedule synchronizations, we can keep those agents from quitting randomly and we can ensure that all the files sync without incident.

Moving the home folder in the cloud
Its worth noting that I am only advocating that you move the Desktop and Documents folder to the cloud, the Library should never be moved to the cloud ever, and your Music and Movies can be moved as long as you are not afraid of the price tag of storing these in the cloud. So lets use Google Drive as the example for this article although you can apply these concepts to any cloud based storage solution.

sudo mv rm -r /Users/Your User Name/Desktop /Users/Your User Name/Desktoptemp 

&&  /Users/Your User Name/Documents /Users/Your User Name/Documentstemp

This command simply renames your Desktop and Documents folder, so that you don’t loose any of those files.The next step is for you to install and configure Google Drive on your computer. Once you have it installed you need to make sure that its fully synchronized. Create a folder in your Google Drive folder called Desktop and Documents. Again these concepts can be translated to any cloud based sync solution.

sudo rm -r /Users/Your User Name/Desktop /Users/Your User Name/Documents 

&& ln -s /Users/Your User Name/Google Drive/Documents /Users/Your User Name/Desktop 

&& ln -s /Users/Your User Name/Google Drive/Desktop /Users/Your User Name/Documents 

&& killall Finder

This is a loaded command, first it removes the local Desktop and Documents folder which you should no longer have since they were moved to the temporary location. Then we are creating a link from the Google Drive Desktop & Documents folder to the respective locations in your home folder. Finally we are restarting the finder. At this point any files you move to Desktop or Documents will be sync’d with Google Drive. Move any of the files from the Desktoptemp and Documentstemp folder to their respective locations i.e.: Desktop & Documents and watch it all start syncing to the cloud.

Keeping it running
The big problem now is making sure that its stable and that there is not a lot of user interaction involved. The Google Drive agent is notorious for silently quitting and failing to sync files. This is a problem. Lets create a script that will run every 30 min to check if the process is still running, and if not have it re-launch it.

#!/bin/bash

if ps ax | grep -v grep | grep "$SERVICE" > /dev/null
then
    echo "$SERVICE service running, everything is fine"
else
    echo "$SERVICE is not running"
    open -a "/Applications/Google Drive.app"
fi

Lets look at this script, basically it checks to see if the Google Drive.app is running, if its not then it launches it. This is great, except for this scenario. What happens if you have multiple users on a computer all with Google Drive running, and one of them has it running the other does not. We need to have a script that works with multiple users too.

#!/bin/bash
SERVICE='Google Drive.app'
 
if (( $(ps x | grep "$SERVICE" | wc -l) < 2 ))
then
    echo "$SERVICE is not running"
    open -a "/Applications/Google Drive.app"
else
    echo "$SERVICE service running, everything is fine"
fi

This script will look for instances less than two, assuming there are two users on a computer and fast user switching is enabled. If not, then the first script will work fine. But if there are simultaneous logged in users then you just change the number 2 in the above script to whatever the total number of logged in users in. For me I always have 2, myself and my wife.

Take this script and save it to a folder called scripts at the root of your HD, so it can be run by all users. Also make sure you chmod it to be executable.

sudo chmod -R 755 /scripts/sync.sh

Scheduling the script
I recommend using LaunchD to manage scheduling on Mac OSX. I use a program called Lingon, you can by this from the App Store, or you can hand code your plist file. In Lingon, you would create a new plist file, I named mine com.jonbrown.drive and I have the script path set in the "What" field. I set it to run every 30 min as you can see here.

If you want to make the plist file yourself you can use the below as a template. I wrote a blog that goes over how to use LaunchD items for your reference.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>KeepAlive</key>
	<false/>
	<key>Label</key>
	<string>com.jonbrown.drive</string>
	<key>LingonWhat</key>
	<string>/scripts/sync.sh</string>
	<key>ProgramArguments</key>
	<array>
		<string>/scripts/grep_disk.sh</string>
	</array>
	<key>RunAtLoad</key>
	<false/>
	<key>StartInterval</key>
	<integer>30</integer>
</dict>
</plist>

With this running at scheduled intervals now the agent can quit, but it will relaunch every 30 seconds if it does. Also this fixes the issue of the files not syncing, usually if a file is modified since the last sync it fails but in this case a file is never more than 30 seconds out of sync. The last issue is scheduling a sync and as you can see here we have it scheduled for every 30 seconds.

Scaling this...
You can do this as many times to as many computers for example I have my Home, Work and Laptop computers all synced and I just did the same process with the same Google Drive account on each one. I hope this helps you out, and makes your life easier!!

The Adaptive Firewall
The most basic task you can do with the firewall is to disable all of the existing rules. To do so, simply run afctl (all afctl options require sudo) with a -d option:

afctl -d

When run, the adaptive firewall’s rules are disabled. To re-enable them, use the -e option:

afctl -e

To remove a specific IP address that has been blacklisted, use the -r option followed by the IP address (rules are enforced by IP)

sudo /usr/libexec/afctl -r ###.###.###.0/24

To add an IP to the blacklist, use the -a option, also followed by the IP

sudo /usr/libexec/afctl -a ###.###.###.0/24

To permanently add a machine to the whitelist, use -w with the IP

sudo /usr/libexec/afctl -w ###.###.###.0/24

Recover a corrupt open directory
10.5, 10.6, and 10.7 have a recover tool that will help you recover a damaged or corrupt ldap directory.

sudo db_recover -h /var/db/openldap/openldap-data/

Reset a corrupt open directory
Note, this should only be done in the event that there is no possible way to recover or restore the OD. This will completely destroy your servers open directory.

sudo slapconfig -destroyldapserver

mkpassdb -dump

Change computer name
Change the computer name over ARD or SSH.

networksetup -setcomputername 

Cleanup files from users computers
We do not allow .torrent files on users computers, so I run this via ARD every morning on the entire network. I later adapted these commands to a script that runs on login.

rm -vrf ~/Downloads/*.torrent
rm -vrf ~/Desktop/*.torrent
rm -vrf ~/Documents/*.torrent

Establish Jailed SSH
This command will allow you to establish a secure connection over ssh with an encrypted key pair.

cat ~/.ssh/id_dsa.pub | ssh root@xx.xx.xx.xx 'cat - >> ~/.ssh/authorized_keys'

Unison
Unison is an amazing utility that runs as a service on OSX that will do two way file syncing over ssh or locally. When setting up Unison you must copy it to /usr/bin/ and then create this directory for it to run.

mkdir ~/.unison

For more information on Unison you should check out this site.

Running A Unison Batch
You can run this command with Unison to start a file syncronization.

unison -batch "/SRC/Dest/" ssh://someuser@xx.xx.xx.xx/Dest/Folder/

Check a user record
Often times its easier to lookup an account and see its attributes in the terminal rather than using Workroup Manager.

dscl -u diradmin -P  /LDAPv3/fully.qualified.domain -read /Users/username

Sync MYSQL between servers
If you need to do a backup from one MYSQL database to another you can use this command to do so. You can also use this as a way to dump a database to a different server.

mysqldump --user= --password=  | ssh @xx.xx.xx.xx -p8286 mysql --user= --password= 

RSYNC to remote server
Sometimes you need to backup files across a network to a different computer this method allows you to do that over a secure ssh connection.

rsync -av -e ssh /SRC/Folder/ --rsh='ssh -p8286' @xx.xx.xx.xx:/DEST/Folder/

Force remove a broken OD replica
OD Replicas can be tricky, over time they can fail and sometimes when a replica fails, and you decommission it, it does not get fully removed on the OD Master. Here is how to update the Master to remove that stubborn old replica record.

/var/db/authserver
mkpassdb -dump
slapconfig -removereplica xx.xx.xx.xx

Fix broken Mobile Account run on the local machine
Mobile accounts sometimes need to be removed from the local computer locally. Here is a simple terminal command to remove a mobile account from a local machine.

dscl . -delete /Users/userName

Change local password with ARD
This is the easiest way to change user passwords on remote computers with ARD. You can also use secure SSH but ARD is much easier.

dscl . -passwd /Users/userName newpass newpass

Enable ARD remotely
How can you enable ARD on a computer system that you have SSH access to? Hers how!

sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -activate -configure -access -on -restart -agent -privs -all

Fix SSL on servers
Sometimes SSL on an OSX Server can break, here is how to get it back up and running for your server and all of the ssl sites.

1. Put the bundle package “gd_bundle.crt” in the /etc/apache2/ directory

sudo pico /etc/apache2/httpd.conf

3. Enter this line in the SSL block

SSLCertificateChainFile "/etc/apache2/gd_bundle.crt"

Fix Apple SUS
This sometimes happens, where the symlinks on the server will break, instead of changing all the clients to point to the other catalog that you mentioned, I changed the symbolic link at the server to point to it instead. In Teminal at the server:

cd /var/db/swupd/html

sudo rm index.sucatalog

sudo -u _softwareupdate ln -s /var/db/swupd/html/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog index.sucatalog

Enable Screen Share (VNC) from Terminal

cd /Library/Preferences
echo -n enabled > com.apple.ScreenSharing.launchd

Remotely Set Volume level on a computer

sudo osascript -e "set Volume 10"

Manually Set SUS in OSX

sudo defaults write /Library/Preferences/com.apple.SoftwareUpdate CatalogURL http://fully.qualified.domain:8088/index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog

Remove Microsoft License
If your not fortunate enough to be using a site license for Microsoft Office then you can remove or revoke a license in the terminal like this.

sudo rm ~/Library/Preferences/Microsoft/Office 2008/Microsoft Office 2008 Settings.plist 
sudo rm /Applications/Microsoft Office 2008/Office/OfficePID.plist

Search and replace in SQL

UPDATE wp_posts SET post_content = replace(post_content,"coolstuff.com","lancelhoff.com")

Change Mailman Password
How to quickly change the mailman password on an OSX Server installation.

cd /usr/share/mailman/bin/

./mmsitepass

3. Change password

Export Mailman Lists
How to export Mailman Lists to text files on an OSX Server installation.

cd /usr/share/mailman/bin/

./list_lists

./list_members FWW-Internal > ~/Desktop/somefile.txt

Reset the localKDC
This is important to do, if you do not do this, you will not be able to bind your computer after an ASR restore to an OSX Directory. Tools like Deploy Studio Server and Casper run these commands for you.

1. Delete all 3 com.apple.kerberos.kdc in the login keychain

sudo rm -fr /var/db/krb5kdc

sudo /usr/libexec/configureLocalKDC

Restart the Wiki Service

sudo serveradmin stop teams
sudo serveradmin start teams

List out the size of folders on the file system

sudo du -d 1 -h -x /Volumes/HDName/

Change the way the dock behaves
Use Suck instead of Scale

defaults write com.apple.dock mineffect -string suck
killall Finder

Report on all activity from a specific user
Good for seeing exactly what users are doing on the server or on their computers.

ps aux | grep "root" | more >> ~/rootreport.txt

Export a list of contacts from MailMan OSX Server

/usr/share/mailman/bin/list_members list_name > saved_subscribers

Restart the Mail Service Remotely

sudo serveradmin stop mail && sudo serveradmin start mail

Start the SSH service on a server remotely

echo "AdminsPassHere" | sudo service ssh start

Enable universal access remotely

osascript -e 'tell application "System Events"' -e 'tell application processes' -e 'key code 28 using {command down, option down, control down}' -e 'end tell' -e 'end tell'

Change email from html to plain text only

defaults write com.apple.mail PreferPlainText -bool TRUE

Change the scrollbars in OSX
This will change how the scrollbar works there will be an up and down arrow, this only works in 10.5, 10.6.

defaults write "Apple Global Domain" AppleScrollBarVariant DoubleBoth

Show hidden files in OSX

defaults write com.apple.finder AppleShowAllFiles TRUE

Eject a stubborn or stuck disk remotely

disktool -e disk#

Change the text in the login window

sudo defaults write /Library/Preferences/com.apple.loginwindow LoginwindowText "Hi, I have missed you!"

Change the dock size

defaults write com.apple.dock largesize -int 512
killall Finder

Change the icon size

defaults write com.apple.finder DesktopViewOptions -dict IconSize -integer 512
killall Finder

Change the desktop tile size

defaults write com.apple.dock tilesize -int 256

Remove Spotlight from OSX
This will destroy spotlight until you run repair permissions.

sudo chmod 0 /System/Library/CoreServices/Spotlight.app
killall Spotlight

Restart the ethernet port

sudo ifconfig en0 down

Copy a file remotely

scp test.zip adminname@fully.qualified.domain:~/Desktop

See the Serial Number of your 10.6 Server
You can obtain the Mac OS X Server serial number (for Snow Leopard) via the command line. At the Terminal on the server itself (or via ssh if you wish), type:

more /etc/systemserialnumbers/xsvr

I found this script which works great at extracting the information that is stored in the plist file in each of the page folders in the Wiki structure. However grabbing the content out of the page.html file stored in each .page folder was what I was looking to do. I wrote a helper script that recursively copies and runs the script with a few modifications and then exports all the data I wanted to CSV. The script then copies the CSV files to the main export folder and then deletes all the files that it created in the WIki Server structure.

Usage

To use this script you must copy the folder and all three of the scripts inside it to the root level of your Server HD. Each script has a variable you must set, once you have set the initial path of your Wiki Deployment and the base URL structure you need to make the files executable. You can do this by

chmod 700 -R /export

this should make the scripts executable. Once done you need to run the run.sh script with sudo. This will trigger the export. This is no where near perfect so I have opened up a GitHub repository for the changes that I have made, and the addition to the helper script that runs these recursively. This also exports content in user blogs as well.

The one challenge I am having is running the script that exports the page.html file content and keeping the encoding at utf-8 so that I don’t get any artifacts or odd characters.

Here are the scripts

Run.sh

#!/bin/bash

##### CONFIGURE HERE ########

# put your full path to your collaboration files

fullpath=/Wiki/wiki/Collaboration

##### END CONFIGURATION #####

mkdir /export/users
mkdir /export/users/blogs
mkdir /export/groups
mkdir /export/groups/blogs
mkdir /export/groups/wikis

for i in `ls $fullpath/Groups`
do
cp /export/export-blog.sh $fullpath/Groups/$i/weblog/
cp /export/export.sh $fullpath/Groups/$i/wiki/

# Export Group Wikis

cd $fullpath/Groups/$i/wiki/
./export.sh
mkdir /export/groups/wikis/$i
cp $fullpath/Groups/$i/wiki/wikipages.csv /export/groups/wikis/$i/
rm $fullpath/Groups/$i/wiki/wikipages.csv
rm $fullpath/Groups/$i/wiki/export.sh

# Export Group Blogs

cd $fullpath/Groups/$i/weblog/
./export-blog.sh
mkdir /export/groups/blogs/$i
cp $fullpath/Groups/$i/weblog/wikipages.csv /export/groups/blogs/$i/
rm $fullpath/Groups/$i/weblog/wikipages.csv
rm $fullpath/Groups/$i/weblog/export-blog.sh

done

for i in `ls $fullpath/Users`
do

# Export User Blogs

cp /export/export-blog.sh $fullpath/Users/$i/weblog/

cd $fullpath/Users/$i/weblog/
./export-blog.sh
mkdir /export/users/blogs/$i
cp $fullpath/Users/$i/weblog/wikipages.csv /export/users/blogs/$i/
rm $fullpath/Users/$i/weblog/wikipages.csv
rm $fullpath/Users/$i/weblog/export-blog.sh

done

exit 0

export.sh

#!/bin/sh - 
#
# Script to extract data from an Apple WikiServer's data store by querying the
# filesystem itself. Creates a 'wikipages.csv' file that's readable by any
# spreadsheeting application, such as Numbers.app or Microsoft Excel.app.
#
# USAGE:   To use this script, change to the WikiServer's pages directory, then
#          just run this script. A file named wikipages.csv will be created in
#          your current directory. For instance:
#
#              cd /Library/Collaboration/Groups/mygroup/wiki  # dir to work in
#              wikipages2csv.sh                               # run the script
#              cp wikipages.csv ~/Desktop                     # save output
#
# WARNING: Since the WikiServer's files are only accessible as root, this script
#          must be run as root to function. Additionally, this is not extremely
#          well tested, so use at your own risk.
#
# Author:  Meitar Moscovitz
# Date:    Mon Sep 22 15:03:54 EST 2008

##### CONFIGURE HERE ########

# The prefix to append to generated links. NO SPACES!
WS_URI_PREFIX=http://my-server.example.com/groups/wiki/

##### END CONFIGURATION #####
# DO NOT EDIT PAST THIS LINE
#############################

WS_CSV_OUTFILE=wikipages.csv
WS_PAGE_IDS_FILE=`mktemp ws-ids.tmp.XXXXXX`

function extractPlistValueByKey () {
    head -n 
      $(expr 1 + `grep -n "$1" page.plist | cut -d ':' -f 1`) page.plist | 
        tail -n 1 | cut -d '>' -f 2 | cut -d '<' -f 1
}

function linkifyWikiServerTitle () {
    echo $1 | sed -e 's/ /_/g' -e 's/&/_/g' -e 's/>/_/g' -e 's/</_/g' -e 's/?//g'
}

function formatISO8601date () {
    echo $1 | sed -e 's/T/ /' -e 's/Z$//'
}

function csvQuote () {
    echo $1 | grep -q ',' >/dev/null
    if [ $? -eq 0 ]; then # if there are commas in the string
        echo '"'"$1"'"'   # quote the value
    else
        echo "$1"         # just output the as it was received
    fi
}

PSTALLY=`ls -l | grep -v ^l | wc -l`

if [ $PSTALLY -gt 4 ] ; then

ls -d [^w]*.page | 
  sed -e 's/^([a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]).page$/1/' > $WS_PAGE_IDS_FILE

fi



echo "Title,ID,Date Created,Last Modified,URI,Content" > $WS_CSV_OUTFILE
while read id; do
    
    cd $id.page
    title="$(extractPlistValueByKey title)"
    created_date="$(formatISO8601date $(extractPlistValueByKey createdDate))"
    modified_date="$(formatISO8601date $(extractPlistValueByKey modifiedDate))"
    link=$WS_URI_PREFIX"$id"/`linkifyWikiServerTitle "$title"`.html
    FILE_DATA=`echo $( /bin/cat page.html ) | tr ',' ' '`
    cd ..
    echo `csvQuote "$title"`,$id,$created_date,$modified_date,`csvQuote "$link"`,"$FILE_DATA" >> $WS_CSV_OUTFILE
done < $WS_PAGE_IDS_FILE
rm $WS_PAGE_IDS_FILE

export-blog.sh

#!/bin/sh -
#
# Script to extract data from an Apple WikiServer's data store by querying the
# filesystem itself. Creates a 'wikipages.csv' file that's readable by any
# spreadsheeting application, such as Numbers.app or Microsoft Excel.app.
#
# USAGE:   To use this script, change to the WikiServer's pages directory, then
#          just run this script. A file named wikipages.csv will be created in
#          your current directory. For instance:
#
#              cd /Library/Collaboration/Groups/mygroup/wiki  # dir to work in
#              wikipages2csv.sh                               # run the script
#              cp wikipages.csv ~/Desktop                     # save output
#
# WARNING: Since the WikiServer's files are only accessible as root, this script
#          must be run as root to function. Additionally, this is not extremely
#          well tested, so use at your own risk.
#
# Author:  Meitar Moscovitz
# Date:    Mon Sep 22 15:03:54 EST 2008

##### CONFIGURE HERE ########

# The prefix to append to generated links. NO SPACES!
WS_URI_PREFIX=http://my-server.example.com/groups/wiki/

##### END CONFIGURATION #####
# DO NOT EDIT PAST THIS LINE
#############################

WS_CSV_OUTFILE=wikipages.csv
WS_PAGE_IDS_FILE=`mktemp ws-ids.tmp.XXXXXX`

function extractPlistValueByKey () {
    head -n 
      $(expr 1 + `grep -n "$1" page.plist | cut -d ':' -f 1`) page.plist | 
        tail -n 1 | cut -d '>' -f 2 | cut -d '<' -f 1
}

function linkifyWikiServerTitle () {
    echo $1 | sed -e 's/ /_/g' -e 's/&/_/g' -e 's/>/_/g' -e 's/</_/g' -e 's/?//g'
}

function formatISO8601date () {
    echo $1 | sed -e 's/T/ /' -e 's/Z$//'
}

function csvQuote () {
    echo $1 | grep -q ',' >/dev/null
    if [ $? -eq 0 ]; then # if there are commas in the string
        echo '"'"$1"'"'   # quote the value
    else
        echo "$1"         # just output the as it was received
    fi
}

ls -d [^w]*.page | 
  sed -e 's/^([a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]).page$/1/' > $WS_PAGE_IDS_FILE

echo "Title,ID,Date Created,Last Modified,URI,Content" > $WS_CSV_OUTFILE
while read id; do
    
    cd $id.page
    title="$(extractPlistValueByKey title)"
    created_date="$(formatISO8601date $(extractPlistValueByKey createdDate))"
    modified_date="$(formatISO8601date $(extractPlistValueByKey modifiedDate))"
    link=$WS_URI_PREFIX"$id"/`linkifyWikiServerTitle "$title"`.html
    FILE_DATA=`echo $( /bin/cat page.html ) | tr ',' ' '`
    cd ..
    echo `csvQuote "$title"`,$id,$created_date,$modified_date,`csvQuote "$link"`,"$FILE_DATA" >> $WS_CSV_OUTFILE
done < $WS_PAGE_IDS_FILE
rm $WS_PAGE_IDS_FILE

I needed a way for my server to notify me by email every time there was a potential problem which results in high CPU usage so that I could mitigate that issue quickly. The server monitor and server admin apps do not allow you to monitor CPU usage and Activity monitor is great as long as you are willing to stand in front of your terminal screen all day. I decided to write a script that would alert me when specific processes started running wild.

#!/bin/bash

processToWatch="PasswordService" # in my case I need to watch convert
emailAddress="me@me.com" # this is my main emailaddress
triggerValue=10 # if the CPU use is above 50% send an email. DO NOT USE a DOT or COMMA!
tempFileName=cpulog # some name of the temp file for the ps, grep data

ps auxww | grep "$processToWatch" | grep -v grep > /Scripts/Logs/$tempFileName
export LINE
(
read LINE
while [ -n "$LINE" ]
do
set $LINE
read LINE
if [ $(echo "$3" | sed -e 's/.[0-9]*//g') -gt $triggerValue ]; then
sudo kill -9 $2;
mail -s "CPU message alert for: $processToWatch" $emailAddress <<-END
This is to inform you that the following process: $processToWatch with PID (Process ID) $2 is now using more than your preset $triggerValue value.

Process: $processToWatch is using: $3 of CPU
The command used is: $11
END
fi
done
)< /Scripts/Logs/$tempFileName

The above script will notify me of an issue with the PasswordService and alert me. This has worked out great for me since I only care about a specific service on one server at a time. I can set the time variable in the script to warn me at a specific interval and I use a cron job to schedule the task. I usually have it running every 5 minutes. If you need help with the cron job you can refer to my past post on scheduling tasks on servers using Cron.

I have been playing around with two pieces of software that promise to help bring back this lost functionality in an easy to use GUI tool and perhaps even restore a little sanity to running a website on Lion. The first application that I reviewed was called VirtualHostX.

VirtualHostX 3.0 is the easiest way to host and share multiple websites on your Mac. It’s the perfect solution for web designers working on more than one project at a time. (Aren’t we all?) No more nesting folders or asking the programmer across the cubicle for help. With VirtualHostX you can easily create and manage Apache virtual hosts with just a few clicks.

The other feature that I love about this tool is that you can share a private webpage or site that you are working on, that is not publicly available and share it with anyone publicly through a secure password protected connection. This is great if you need to show people updates of your site and their not on the local subnet. This tool allows you to code custom directives (If you need a list you can check out my last post Missing Manual). 

Out of the box this product works with popular platforms like WordPress and it uses the built in Apache that comes with OSX. Alternatively you can even set it to manage any instance of apache on your server.

Lastly you can even backup the changes that it makes to your system so that you can performa  seamless migration or just for your own peace of mind. I love this software and its an amazing alternative to using the Server app.

The other tool that I found that handles Apache administration on 10.7 is WebMon. Webmon does not look as cool as VirtualHostX however it does have greater support for Custom Directives out of the box in the form of GUI interface.

WebMon configures OS X’s built-in web server to support server-side includes, execCGI, PHP, SSL (including support for inserting Intermediate CA certs) and WebDAV, for multiple domains running on the same server.

With WebDAV turned on, your web server acts like an iDisk, allowing you to connect to the WebDAV folder remotely, securely, and directly from the Finder, so you can save, share, and distribute your files and folders. You can also use the WebDAV folder to share your iCal calendars.

WebMon also helps you set up the web server so that you can monitor its log file from a remote machine. WebMon is able to help you monitor any number of web servers from a single remote machine.

With WebMon you can setup and manage SSL Certificates, turn on CGI Support and much much more. This tool certainly restores almost all of the lost functionality . If you run multiple Web Servers than you might also like its built in monitoring service that makes sure that Apache is running soundly on other systems.

The great thing about both of these solutions is that they work well together, so you can use both or one of them but for the beginner web server administrator these tools restore a little more control when it comes to Apache administration.

I hope that you all found this article and walkthrough educational, as always please feel free to interact with me by posting questions and comments and I will answer them as best as I can. If you feel like any of this is wrong or could be improved upon also please leave a comment below, thanks!

The purpose of this entry is to talk about 10.7 server and show you how to accomplish everything that you could accomplish from the Server Admin application through commands using terminal or edits to system files in the operating system. Everything below requires that you be logged in as the root user on the server in order to avoid permission issues.

How to enable PHP 
Run this command to check if PHP is enabled on 10.7 server.

cat /etc/apache2/httpd.conf|grep libphp5.so

If the output is

LoadModule php5_module libexec/apache2/libphp5.so

and not

#LoadModule php5_module libexec/apache2/libphp5.so

then PHP is enabled. If it is the other way around with a # in the beginning of the line you can just edit the httpd.conf file manually with

sudo pico /etc/apache2/httpd.conf 

and remove the bracket manually and then restart the web server with

sudo apachectl restart

Alternatively you can also enable this via a checkbox in the terrible server.app in 10.7.

How to change the default file type 
By default the landing page on all new sites is index.html if you would like to change this or the order in which a webpage searches for the index page then you need to change the default file type.

To do this edit the configuration file appropriate to your site name. Meaning you have to have already configured a site in the 10.7 server.app program once you have a site then you need to edit the site configuration file. If your site was called apple.com then your site configuration would be in /etc/apache2/sites/apple.com.conf or something like that.

You need to edit that file

pico /etc/apache2/sites/nameofyoursite.conf

look for the following in the file

<IfModule mod_dir.c>
    DirectoryIndex index.html
</IfModule>

If you want to change the main page to index.php instead of index.html then replace index.html with index.php. If you want to add it as a secondary load page then you can change it to this.

<IfModule mod_dir.c>
    DirectoryIndex index.html index.php
</IfModule>

once done save and restart apache.

sudo apachectl restart

How to enable .htaccess 
If you are going to be using mod_rewwrite at all for redirects or pretty permalinks (which is very common now) then you need to have this enabled. Again as stated before you have to have a site setup on the server through the server.app program. Once done locate your configuration file as outlined above and make the following changes.

pico /etc/apache2/sites/nameofyoursite.conf

Once your in the file look for something that looks similar to the following.

<Directory "/Users/yourname/Sites/">
     Options Indexes +MultiViews
     AllowOverride All
     Order allow,deny
     Allow from All
</Directory>

It won’t look exactly the same but what you want to do is replace it with what you see above that will enable the .htaccess or mod_rewrite the line of code that actually does this is the “AllowOverride All” command.

How to enable WebDav
To configure WebDAV Sharing for such users, follow these instructions before enabling any WebDAV share points.

Note: The instructions in this article include editing configuration files. You must have root access to edit these files. You should make a backup copy of each file prior to editing it.

This step is optional but highly recommended: Acquire and install a trusted SSL certificate, and use Server App to configure Web Service to use the certificate. You can use the server’s default, self-signed certificate for WebDAV Sharing, but iWork and other applications may warn that the certificate is “invalid”.

You need to edit the following configuration file

pico /etc/apache2/httpd_webdavsharing.conf

Find the line “AuthType Digest” change Digest to Basic. This makes WebDAV Sharing use Basic authentication, which is required for Active Directory users.

Now edit this configuration file

pico /etc/apache2/webapps/com.apple.webapp.webdavsharing.plist

find these lines

<key>sslPolicy</key>
<integer>0</integer>

Change the 0 to 1. This makes WebDAV Sharing require SSL, which is the only secure way to use Basic authentication. Advise users to configure the iWork clients on their iOS devices with an “https” WebDAV URL, like: https://example.com/webdav

How to enable the directory listing 
Again as stated before you have to have a site setup on the server through the server.app program. Once done locate your configuration file as outlined above and make the following changes.

You need to edit that file

pico /etc/apache2/sites/nameofyoursite.conf

find the words “AllowOverride” in that block where these words are you need to add this line. This line may already be in your file but it may be different simply update it to reflect these changes

Options -Indexes FollowSymLinks

How to enable SSI
If you need to use Server Side Includes in your scripts or website files then do the following to enable it.

sudo pico /etc/httpd/httpd.conf

look for these lines

# AddType text/html .shtml
# AddHandler server-parsed .shtml

Uncomment those 2 lines (remove the # in front of each of them). Now look in the same file for the following

Options FollowSymLinks

Add “Includes” to the 2nd line so it looks like

Options FollowSymLinks Includes

save the file and restart apache

sudo apachectl restart

How to enable VHOSTS
VHOSTS or Virtual Hosts enable you to have multiple domain names mapped to the same site or IP address. To enable this edit the httpd.conf file

sudo pico /etc/apache2/httpd.conf 

find this line

#Include /private/etc/apache2/extra/httpd-vhosts.conf

change it to

Include /private/etc/apache2/extra/httpd-vhosts.conf

this will effectively enable VHOSTS. Now you should restart apache.

sudo apachectl restart

How to enable CGI
Again as stated before you have to have a site setup on the server through the server.app program. Once done locate your configuration file as outlined above and make the following changes.

pico /etc/apache2/sites/nameofyoursite.conf

Once your in the file look for something that looks similar to the following.

     Options Indexes +MultiViews

It won’t look exactly the same but what need to do is add “-ExecCGI” after “+MultiViews” it should look something like this.

     Options Indexes +MultiViews -ExecCGI

This will enable CGI and allow you to run CGI scripts in Apache. Now you should restart apache.

sudo apachectl restart

How to enable Logging
This one boggled my mind, by default website logging is not enabled and again there is no way to enable it in the GUI. You will want to have this enabled to catch errors and fix faulty code. To enable this again we are assuming you already have a site configured with the server.app program. Once done locate your configuration file as outlined above and make the following changes.

pico /etc/apache2/sites/nameofyoursite.conf

find the line “DocumentRoot”, Under that line paste the following

CustomLog "/var/log/apache2/access_log" combinedvhost
ErrorLog "/var/log/apache2/error_log"

it should now look like this

DocumentRoot "/path/to/your/website/"
CustomLog "/var/log/apache2/access_log" combinedvhost
ErrorLog "/var/log/apache2/error_log"

Now you should restart apache.

sudo apachectl restart

How to add a domain alias
This is a common thing that most web admins do to map domains to a single site. This again has been removed from the functionality of the server.app on 10.7 server but is a pretty easy to add. To enable this again we are assuming you already have a site configured with the server.app program. Once done locate your configuration file as outlined above and make the following changes.

pico /etc/apache2/sites/nameofyoursite.conf

in the site definition file, look for a line that says

ServerName example.com
ServerAlias www.example.com

where example.com is the domain of your site. You can have more than one alias, just separate them by a spaces on the same line like so.

ServerName example.com
ServerAlias www.example.com alias2.example.com alias3.example.com

Now you should restart apache.

sudo apachectl restart

How to restore factory settings to 10.7 Web Service
This one is important. As stated above you should be backing up these config files before you edit them and then making your changes. In the event that something went wrong you can always reset them back to the original settings.

Run this command

sudo serveradmin command web:command=restoreFactorySettings

I got this command by calling Apple directly they also suggested restarting the machine after the restore command, once the computer is back up turn off and then turn on web service to ensure it is working propperly.

Conclusion
All of these commands allow you to leverage Apache and accomplish the tasks that were once easy to accomplish with the Server Admin tool in 10.6 server. There are two options here, learn to love the command line or do not upgrade to 10.7 Lion. Apple is streamlining their GUI interfaces for their tools however there is still power under the hood. Do not be afraid to re-configure these systems Apache, PHP and MYSQL can be installed, modified and improved all from the command line and in some cases they work better after you do. Its not time to quit in my opinion its time to roll up our sleeves and start learning the core of what makes an OSX server truly great and that starts with understanding the open source software that comes bundled with them.

I hope that you all found this article and walkthrough educational, as always please feel free to interact with me by posting questions and comments and I will answer them as best as I can. If you feel like any of this is wrong or could be improved upon also please leave a comment below, thanks!

“I just setup a new server and within days we were on a corporate email blacklist, I contacted the company in question and asked why are we on your blacklist, why won’t you deliver our email. They shared with me an email log of thousands of emails being sent from my mail server through several legitimate email accounts. I ensured that my server was not an open relay so I asked these users, if they had indeed sent this many emails in one shot without any kind of unsubscribe link in the footer of their email. They had! I was so shocked, now what do I do?”

This is an uncomfortable and very perilous position. You want to allow your users to send email to get their job done however you as a systems administrator need to comply with the “Can Spam Act” passed by the FCC to ensure that email continues to flow. You also have companies out there who will block you for violating this act as a precaution on their part. All the while your users can not be bothered to learn about proper email procedures.

In my experience the only thing you can do at this point is to limit how many emails are allowed to be sent at any given time. If you are using OSX Server for Mail or Postfix for Sendmail then this walkthrough will talk about how to limit email recipients and stay off those dreaded blacklists.

Here are the basics that you should know, the following are all settings that can be added to the /etc/postfix/main.cf file of your postfix setup.

smtpd_recipient_limit (default 1000) parameter controls how many recipients the SMTP server will take per message delivery request. You can’t restrict this to a to/cc/bcc field – it’s for all recipients. For that you’d have to use a regular expression in header_checks to arbitrarily limit the length of each header to something reasonable.

smtpd_recipient_overshoot_limit (default 1000) The number of recipients that a remote SMTP client can send in excess of the hard limit specified with smtpd_recipient_limit, before the Postfix SMTP server increments the per-session error count for each excess recipient.

smtpd_hard_error_limit (default 20) parameter to know at what number of errors it will disconnect.

So you technically need to consider the 3 values here which affect both inbound & outbound mail. Then there’s the throttling tools.

smtpd_client_recipient_rate_limit (default: 0 no limit) The maximum number of recipient addresses that an SMTP client may specify in the time interval specified via anvil_rate_time_unit (default: 60s -careful adjusting this affects other things)” and note that this is “regardless of whether or not Postfix actually accepts those recipients” Those over will receive a 450 4.7.1 Error: too many recipients from [the.client.ip.address] It’s up to the client to deliver those recipients at some later time.

smtpd_client_connection_rate_limit (default: 0) The maximal number of connection attempts any client is allowed to make to this service per time unit. The time unit is specified with the anvil_rate_time_unit configuration parameter.

smtpd_client_message_rate_limit (default: 0) The maximal number of message delivery requests that any client is allowed to make to this service per time unit, regardless of whether or not Postfix actually accepts those messages. The time unit is specified with the anvil_rate_time_unit configuration parameter.

The purpose of these features are to limit abuse, as opposed to regulating legitimate mail traffic, but I use them that way in order to mitigate spam blacklisting. In my organization we limit the recipients from one email to 25 you can see the code from my sample /etc/postfix/main.cf. If your file does not have these values you can add them to the bottom of the file.

smtpd_recipient_limit = 50
smtpd_recipient_overshoot_limit = 51
smtpd_hard_error_limit = 20
smtpd_client_recipient_rate_limit = 50
smtpd_client_connection_rate_limit = 10
smtpd_client_message_rate_limit = 25
default_extra_recipient_limit = 50
duplicate_filter_limit = 50
default_destination_recipient_limit = 50
smtp_destination_recipient_limit = $default_destination_recipient_limit

Once done you need to restart postfix

sudo postfix reload

I hope that you all found this article and walkthrough educational, as always please feel free to interact with me by posting questions and comments and I will answer them as best as I can. If you feel like any of this is wrong or could be improved upon also please leave a comment below, thanks!

Installation & Configuration

1. Download and install the 64-bit 10.6+ version of MYSQL installer package together with the startup files here.

http://dev.mysql.com/downloads/mysql/

2. Mount the Disk Image (I mean open/double-click the DMG file) and install MySQL server by double-clicking the PKG file (in my case mysql-5.5.14-osx10.6-x86_64.pkg) and follow onscreen instructions. ( It will ask for Master password, as it installs MySQL server in /usr/local )

Current latest version is 5.5.14 which I’ll be using to install on my server.

Open the DMG and you will see that the first item is the MySQL software, the 2nd item allows MySQL to start when the Mac is booted and the third is a System Preference that allows start/stop operation and a preference to enable it to start on boot. Run all of these.

Once the installs are done you can start the mysql server right from the System Preferences which has a new preference in the “Other” category called “MySQL” click start and now it is running.

To find the MySQL version from the terminal, type at the prompt

/usr/local/mysql/bin/mysql -v

If you got the error: ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/tmp/mysql.sock’

then mysql was not started, go back to the System Preference and start the database.

3. Run the following commands

cd /usr/local/mysql
cp /usr/local/mysql/support-files/my-small.cnf /private/etc/my.cnf
open -e /private/etc/my.cnf

replace “/tmp/mysql.sock” with “/var/mysql/mysql.sock” at two places near the top.
Create a folder called “mysql” (if you don’t already have one) in the /var directory with the right permissions:

cd /var
mkdir mysql
sudo chown -R mysql mysql 
sudo chmod 775 mysql

This command will circumvent the dreaded mysql 2002 socket error.

sudo mkdir /var/mysql
sudo ln -s /tmp/mysql.sock /var/mysql/mysql.sock

4. Create your alias, this is important so that you can run MYSQL queries through the terminal.

alias mysql /usr/local/mysql/bin/mysql
alias mysqladmin /usr/local/mysql/bin/mysqladmin

optionally you can edit the ~/.profile file to make your aliases (This should be done as root)

pico ~/.profile

then add this line below

export PATH=/usr/local/mysql/bin:$PATH

*Please note /usr/local/mysql is only symlink to /usr/local/mysql-5.5.14-osx10.6-x86_64 which means when you upgrade to new version symlink will be changed to point to new version but won’t be deleting the older version. However you need to copy your data directory to new location to make sure your existing databases are intact post upgrade.

5. Set the master MYSQL password, there are 2 ways to do this one is a regular way and the other provides additional security and disables all other access

Regular Way

mysqladmin -u root password 'yourpasswordhere'

** use the single quotes. Then when login to mysql to test your password

mysql -u root -pyourpasswordhere

Secure Way

sudo mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MySQL to secure it, we'll need the current
password for the root user. If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.
Enter current password for root (enter for none):

Go ahead and just hit enter if this is a new installation and no password currently exists, follow the prompts to set up a new root password – this is a root password just for mysql separate from the root password of OS X you should set this.

You also get asked about:

• Removing anonymous users?
• Disallow root login remotely?
• Remove test database and access to it?
• Reload privilege tables now?
• If this is a new installation you can just answer yes to the questions.

Once the root user and password is set, you have to interact with mysql with the username and password, so access via command line is (note that there is no space between -p and the password)

mysql -u root -p[password]

Now that you have MYSQL running you need to start an instance or a main profile for MYSQL to run. I have found the easiest way to do this is to install PHPMYADMIN and since most people in my opinion (Again novice to intermediate MYSQL user here) use this great tool to navigate around MYSQL on a daily basis. Here is a brief walkthrough on how to install and configure PHPMYADMIN on 10.7 Lion Server

Installation & Configuration

1. Change the socket location in your PHP configuration by editing the php.ini file. You need to do a search and replace here. Search and replace all instances of

/var/mysql/mysql.sock

with

/tmp/mysql.sock

Once done you should be able to run the following command and it should reflect the new updated values you just applied.

grep .default_socket /etc/php.ini

while editing the php.ini file you need to comment out or enable the following extensions.

extension=php_mysql.dll
extension=php_mysqli.dll

To check your work again you can run this command to ensure they are enabled.

grep mysql /etc/php.ini|grep ext

Once done restart Apache

sudo apachectl restart

2. Download PHPMYADMIN to the default web directory in Lion

http://www.phpmyadmin.net/home_page/index.php

The full path is

/Library/Server/Web/Data/Sites/Default

I put my PHPMYADMIN in a folder called PHP so

/Library/Server/Web/Data/Sites/Default/PHP

and I could then browse to it by going to

http://server.domain.name/PHP/

this is assuming that you have already configured or turned on web services which I will not go into here since it is a very basic step. I will write a more in depth article and how to on the complexities of running an 10.7 web server in the future however.

Run this command on the PHP Config folder

chmod o+w /Library/Server/Web/Data/Sites/Default/PHP/config

3. Now we are ready to run the set up by going to

http://localhost/PHP/setup

The new server to be configured is the localhost, click new server and then the only other configurations are the local mysql user and the password.

Add in the username, by default “root” is assumed, add in the password, click on save and you are returned to the previous screen.

Make sure you click on save, then a config.inc.php is now in the /config directory, move this file to the root level of /phpmyadmin and then remove the empty /config directory.

Now going to http://localhost/PHP/ will now allow you to interact with your mysql databases.

I hope that you all found this article and walkthrough educational, as always please feel free to interact with me by posting questions and comments and I will answer them as best as I can. If you feel like any of this is wrong or could be improved upon also please leave a comment below, thanks!