What’s in My Exchange DR tool BoxWhat’s in My Exchange DR tool Box
Exchange Disaster Recovery Exchange Disaster Recovery Fundamentals – Kind of toolishFundamentals – Kind of toolish
HA + Recovery Options HA + Recovery Options
Dialtone RecoverDialtone Recover
Setup / BlaSetup / Bla
Legacy Exchange mailbox serversLegacy Exchange mailbox serversFailover Cluster Failover Cluster
Exchange 2007 Mailbox roleExchange 2007 Mailbox roleSingle Copy Cluster (SCC)Single Copy Cluster (SCC)
Clustered Continuous Replication (CCR)Clustered Continuous Replication (CCR)
Local Continuous Replication (LCR)Local Continuous Replication (LCR)
Standby Continuous Replication (SCR)Standby Continuous Replication (SCR)
Exchange 2010 Mailbox roleExchange 2010 Mailbox roleDatabase Availability Group (DAG) Database Availability Group (DAG)
Dialtone RecoveryDialtone RecoveryPurpose is to give you time to fix or repair a Purpose is to give you time to fix or repair a databasedatabase
How toHow toMove the active database and log files to a Move the active database and log files to a temp locationtemp location
Mount the databases and say yes when it Mount the databases and say yes when it asks you if you want to create a new asks you if you want to create a new databasedatabase
Restore or recover the databaseRestore or recover the database
Swap the recovered and dialtone databases Swap the recovered and dialtone databases
Play all of the dialtone database information Play all of the dialtone database information into the recovered databaseinto the recovered database
Things to noteThings to noteUsers get new email but they cannot see old Users get new email but they cannot see old stuff in Outlook at the same time.stuff in Outlook at the same time.
Users can see new email in OWA, and old Users can see new email in OWA, and old email in Outlook at the same timeemail in Outlook at the same time
Setup / BlaSetup / Bla
2000 and 2003 setup /disasterrecovery will restore 2000 and 2003 setup /disasterrecovery will restore all of the settings from AD to a serverall of the settings from AD to a server
Most everything is in ADMost everything is in AD
2007 setup /recoverserver will restore all of the 2007 setup /recoverserver will restore all of the settings from AD to a serversettings from AD to a server
Not everything is in AD – it is different for each server roleNot everything is in AD – it is different for each server role
The Databases – Email and QueuesThe Databases – Email and Queues
IIS metabase on the CAS serverIIS metabase on the CAS server
UM prompts on the UM serverUM prompts on the UM server
Registry and XML customizationsRegistry and XML customizations
Message Tracking logsMessage Tracking logs
2007 clusters have setup /recoverCMS that will restore 2007 clusters have setup /recoverCMS that will restore cluster settings to a nodecluster settings to a node
Asynchronous Continuous Asynchronous Continuous ReplicationReplication
Primary ServerPrimary Server
Mailbox DatabaseMailbox Database
Standby ServerStandby Server
Mailbox DatabaseMailbox Database
Transaction Log 01Transaction Log 01Transaction Log 01Transaction Log 01
Transaction Log 02Transaction Log 02Transaction Log 02Transaction Log 02
Transaction Log 03Transaction Log 03Transaction Log 03Transaction Log 03
Transaction Log 04Transaction Log 04Transaction Log 04Transaction Log 04
Transaction Log 05Transaction Log 05Transaction Log 05Transaction Log 05
Transaction Log 01Transaction Log 01Transaction Log 01Transaction Log 01
Transaction Log 02Transaction Log 02Transaction Log 02Transaction Log 02
Transaction Log 03Transaction Log 03Transaction Log 03Transaction Log 03
Transaction Log 04Transaction Log 04Transaction Log 04Transaction Log 04
Transaction Log 05Transaction Log 05Transaction Log 05Transaction Log 05
Exchange Tools - FundamentalsExchange Tools - Fundamentals
Exchange Tools – that I go back tooExchange Tools – that I go back tooRelatively harmless toolsRelatively harmless tools
Dsadiag – Test Exchange to AD connectionDsadiag – Test Exchange to AD connection
Dcdiag + NetDiag – Check AD functionalityDcdiag + NetDiag – Check AD functionality
Nltest – Old school Trust CheckingNltest – Old school Trust Checking
Mdbvu32 + Outlook Spy + MFCMapi – Look at the raw databaseMdbvu32 + Outlook Spy + MFCMapi – Look at the raw database
Winroute – Check active 2003 routingWinroute – Check active 2003 routing
Replmon – Public folder toolReplmon – Public folder tool
ExTRA – Best Practice Analyzer Family of toolsExTRA – Best Practice Analyzer Family of tools
MPS_Reports – Run a heap of toolsMPS_Reports – Run a heap of tools
ExMerge – Ripe mailbox out of the StoreExMerge – Ripe mailbox out of the Store
ExMon – Check Client usageExMon – Check Client usage
PFdavAdmin – Permissions and PF stuffPFdavAdmin – Permissions and PF stuff
ADModify – Bulk Check and Change AD stuffADModify – Bulk Check and Change AD stuff
Escalation to the next level, and or a PSS callEscalation to the next level, and or a PSS call
Dangerous tools – Don’t touch these unless you mustDangerous tools – Don’t touch these unless you mustNtdsutil – Exeutil for ADNtdsutil – Exeutil for AD
Isinteg – Application level EDB toolIsinteg – Application level EDB tool
ADSIEdt + LDP – Muck with Raw ADADSIEdt + LDP – Muck with Raw AD
Eseutil – Data level EDB toolEseutil – Data level EDB tool
Exchange Maintenance ToolsExchange Maintenance ToolsDsadiagDsadiag
W2K Support ToolsW2K Support Tools tool tool
Used for diagnosing directory & binding Used for diagnosing directory & binding problemsproblems
Where’d that GC go?Where’d that GC go?
What GCs are actually up and answering What GCs are actually up and answering queries?queries?
See KB article Q279423 for detailsSee KB article Q279423 for details
Exchange Maintenance ToolsExchange Maintenance ToolsDcdiag + NetdiagDcdiag + NetdiagWindows Support tools toolsWindows Support tools tools
Used for Active Directory, DNS Used for Active Directory, DNS & Domain troubleshooting& Domain troubleshooting
Is DNS configured correctly for ADIs DNS configured correctly for AD
Are all of the FSMO roles present Are all of the FSMO roles present and reachableand reachable
Why can’t users log on?Why can’t users log on?
Why am I seeing GC/DC errors in Why am I seeing GC/DC errors in the event log?the event log?
What DCs or GCs are visible?What DCs or GCs are visible?As opposed to the ones you As opposed to the ones you thinkthink are visibleare visible
Check IP security policiesCheck IP security policies
Check LDAP bindingCheck LDAP binding
Repair missing or busted Machine Repair missing or busted Machine accountaccount
Exchange Maintenance ToolsExchange Maintenance ToolsNltestNltest
W2K resource kit tool – Old W2K resource kit tool – Old School but still useableSchool but still useable
Used for Domain Trust Used for Domain Trust troubleshootingtroubleshooting
What domains does this server What domains does this server TrustTrust
Which DC is being used in the Which DC is being used in the trusted domaintrusted domain
Determine if a server set as a GC Determine if a server set as a GC is really a GCis really a GC
Exchange Maintenance ToolsExchange Maintenance ToolsMdbvu32 + Outlook Spy + MFCMapiMdbvu32 + Outlook Spy + MFCMapi
Non-invasive MAPI mailbox Non-invasive MAPI mailbox viewerviewer
Browse MAPI hierarchyBrowse MAPI hierarchy
Allows you to manage mailbox Allows you to manage mailbox rules and MTA queue messagesrules and MTA queue messages
What’s all that crap in the What’s all that crap in the system attendant mailbox?system attendant mailbox?
What rules are in effect on a What rules are in effect on a mailbox?mailbox?
Delete bad items from the inboxDelete bad items from the inbox
Outlook Spy is Third Party Outlook Spy is Third Party geared for the Developergeared for the Developer
Kill Junk mailKill Junk mail
Kill RulesKill Rules
Exchange Maintenance ToolsExchange Maintenance ToolsWinroute + RemoitorWinroute + Remoitor
Product CD tool = WinrouteProduct CD tool = Winroute
Call PSS = RemonitorCall PSS = Remonitor
Link state routing diagnostic Link state routing diagnostic tooltool
What routes are known to the What routes are known to the routing engine?routing engine?
Which ones are up? Which Which ones are up? Which ones are down?ones are down?
When was the last routing When was the last routing update?update?
Inject new routes, and or Inject new routes, and or delete routes with out a delete routes with out a restartrestart
Exchange Maintenance ToolsExchange Maintenance ToolsReplmonReplmon
W2K product CD W2K product CD tooltool
Replication Replication monitormonitor
When was the last When was the last update this server update this server sent?sent?
When was the last When was the last update received?update received?
Why the $X#@! Why the $X#@! isn’t replication isn’t replication working?working?
AD was new in 2000 and 2003. Exchange was the AD was new in 2000 and 2003. Exchange was the first App to really use AD. Exchange Admins had first App to really use AD. Exchange Admins had to be good at trouble shooting ADto be good at trouble shooting AD
Exchange Maintenance ToolsExchange Maintenance ToolsExTRAExTRA
Exchange Trouble Shooter FamilyExchange Trouble Shooter FamilyExBPA – Best Practice AnalyzerExBPA – Best Practice Analyzer
ExDRA – Disaster Recovery AssistantExDRA – Disaster Recovery Assistant
ExDTA – Database Trouble ShooterExDTA – Database Trouble Shooter
ExMFT – Mail Flow Trouble ShooterExMFT – Mail Flow Trouble Shooter
ExPTAExPTA - Performance Trouble Shooter - Performance Trouble Shooter
Think of these tools as PSS and the Product group Think of these tools as PSS and the Product group in a boxin a box
Exchange Maintenance ToolsExchange Maintenance ToolsMPS_ReportsMPS_Reports
PSS tool – download from MicrosoftPSS tool – download from Microsoft
MPS_Reports comes in flavors; Exchange, AD, SQL, MPS_Reports comes in flavors; Exchange, AD, SQL, and so on. Each Flavor checks different thingsand so on. Each Flavor checks different things
Run this tool when you cannot access a server it Run this tool when you cannot access a server it dumps a huge amount of information to a CAB file dumps a huge amount of information to a CAB file that someone can send youthat someone can send you
Event logsEvent logs
Netdiag + Dcdiag Netdiag + Dcdiag
Dumps GPOsDumps GPOs
Runs ExBPARuns ExBPA
ExDumpExDump
And moreAnd more
Exchange Maintenance ToolsExchange Maintenance ToolsExMergeExMerge
Download from MicrosoftDownload from Microsoft
Requires a MAPI install, so it is best to run from a ClientRequires a MAPI install, so it is best to run from a Client
Used to Extract or place data in a mounted databaseUsed to Extract or place data in a mounted database
Need Send as and Receive as rights for the mailboxNeed Send as and Receive as rights for the mailbox
Can Extract a single item from all mailbox – Think virus or Can Extract a single item from all mailbox – Think virus or Accidental carrier limiting emailAccidental carrier limiting email
Exchange Maintenance ToolsExchange Maintenance ToolsExMonExMon
Download from MicrosoftDownload from Microsoft
Determine which users are killing a server, why, and where Determine which users are killing a server, why, and where they livethey live
Exchange Maintenance ToolsExchange Maintenance ToolsPFdavAdminPFdavAdmin
Download from MicrosoftDownload from Microsoft
Play with Permissions on Public Play with Permissions on Public FoldersFolders
ExportExport
ViewView
ChangeChange
Bulk stuffBulk stuff
Push replicationPush replication
Play with Permissions onPlay with Permissions onForms LibrariesForms Libraries
Recover Deleted Items in PFRecover Deleted Items in PF
Exchange Maintenance ToolsExchange Maintenance ToolsADModifyADModify
Download from Microsoft or Download from Microsoft or CodePlexCodePlex
Play with AD on at a Bulk LevelPlay with AD on at a Bulk LevelSelect via LDAP querySelect via LDAP query
Change heaps of settingsChange heaps of settings
Check the Inherit PermissionsCheck the Inherit PermissionsCheck Box for everyoneCheck Box for everyone
This was cool until PowerShellThis was cool until PowerShellCame outCame out
There are different versions thatThere are different versions thatHave different PowersHave different Powers
Other ToolsOther Tools
Notepad / Txtpad – Read Files and see what's in Notepad / Txtpad – Read Files and see what's in themthem
WinDiff – Compare two files and see what is WinDiff – Compare two files and see what is differentdifferent
CSVDE – Import and Export from AD via CSVCSVDE – Import and Export from AD via CSV
LDIFDE LDIFDE – – Import and Export from AD via LDIF Import and Export from AD via LDIF
DNSLint – Check some DNS issues (Rangers love DNSLint – Check some DNS issues (Rangers love this)this)
https://www.testexchangeconnectivity.com/ - Test - Test OutlookOutlook
JetStress – Test the Database, melt drivesJetStress – Test the Database, melt drives
DNSDiag – Like Netdiag, checks DNSDNSDiag – Like Netdiag, checks DNS
Telnet – Check if Ports are answeringTelnet – Check if Ports are answering
PerfMon – Checkum big Perf CountersPerfMon – Checkum big Perf Counters
Event Viewer – Start here pleaseEvent Viewer – Start here please
Exchange Maintenance ToolsExchange Maintenance Toolsntdsutil – This can ruin your careerntdsutil – This can ruin your career
Installed by Windows serverInstalled by Windows server
Management utility for ADManagement utility for ADWhich server is the schema master?Which server is the schema master?
Which DCs are online?Which DCs are online?
Role passing & seizure Role passing & seizure
Defrag the AD databaseDefrag the AD database
Exchange Maintenance ToolsExchange Maintenance ToolsIsinteg – Please have PSS hold your Isinteg – Please have PSS hold your handhandExchange product CD toolExchange product CD tool
Verifies logical / application level Verifies logical / application level structure of databasestructure of database
Mailboxes, folders, message structuresMailboxes, folders, message structures
Brief digression about logical vs. physicalBrief digression about logical vs. physical
Generally safe to runGenerally safe to runAlways safe unless you use the “-fix” Always safe unless you use the “-fix” switchswitch
Requires significant processing timeRequires significant processing time~ 4GB/hr processing rate~ 4GB/hr processing rate
May require multiple passesMay require multiple passesSome errors have prerequisites that have to Some errors have prerequisites that have to be fixed firstbe fixed first
It fixes more each time it is run up to It fixes more each time it is run up to a pointa point
Exchange Maintenance ToolsExchange Maintenance ToolsIsinteg cont’dIsinteg cont’d
When to run itWhen to run itWhen Microsoft tells you toWhen Microsoft tells you to
After an offline restoreAfter an offline restore
When you suspect logical When you suspect logical database corruptiondatabase corruption
Opening a particular Opening a particular mailbox/message makes Outlook mailbox/message makes Outlook crash or diecrash or die
Messages or folders disappearMessages or folders disappear
Item counts are offItem counts are off
When you’re curiousWhen you’re curiousBut only on a non-production But only on a non-production machinemachine
Exchange Maintenance ToolsExchange Maintenance ToolsADSIedt + LDPADSIedt + LDP
Windows Support toolsWindows Support toolsUse this When Microsoft or Use this When Microsoft or someone who's knows what they someone who's knows what they are doing tells you to, and holds are doing tells you to, and holds your handyour hand
There is no undoThere is no undo
Work with Raw object in ADWork with Raw object in ADDelete – Could get you firedDelete – Could get you fired
Modify – Could get your firedModify – Could get your fired
Read - HarmlessRead - Harmless
Write – Not safeWrite – Not safe
Verify – HarmlessVerify – Harmless
Exchange Maintenance ToolsExchange Maintenance ToolsEseutilEseutil
Exchange product CD toolExchange product CD tool
Verifies physical structure of databaseVerifies physical structure of databasePages, pointers, long values, LV tablesPages, pointers, long values, LV tables
Has multiple modesHas multiple modesSome are safe: Some are safe:
/g: integrity check/g: integrity check
/m: dumps various/m: dumps various interesting bits interesting bits
/k: page checksum /k: page checksum verification verification
/y: copy/y: copy
Exchange Maintenance ToolsExchange Maintenance ToolsEseutil cont’dEseutil cont’d
Some modes are semi-safeSome modes are semi-safe/d does an offline defrag/d does an offline defrag
Seldom necessarySeldom necessaryRequires the DB to be dismountedRequires the DB to be dismounted
/r initiates recovery/r initiates recoveryVerifies DB for consistencyVerifies DB for consistencyPlays back sequence of log filesPlays back sequence of log filesWill either fail (DB untouched) or succeed (DB Will either fail (DB untouched) or succeed (DB consistent)consistent)
Some are for emergency use only:Some are for emergency use only:DATA LOSSDATA LOSS/createstm forces creation of a new, but /createstm forces creation of a new, but matching, STM file (2000 and 2003 only)matching, STM file (2000 and 2003 only)/p initiates a repair; missing or damaged /p initiates a repair; missing or damaged pages may be truncatedpages may be truncated
Exchange Maintenance ToolsExchange Maintenance ToolsEseutil cont’dEseutil cont’d
When to run it depends on modeWhen to run it depends on modeWhen Microsoft tells you toWhen Microsoft tells you to
When you suspect physical database When you suspect physical database corruptioncorruption
Event ID -1018 or relatedEvent ID -1018 or related
Database reported as inconsistentDatabase reported as inconsistent
Store crashes or refuses to mount a Store crashes or refuses to mount a particular DBparticular DB
Questions?Questions?