+ All Categories
Home > Data & Analytics > 10 Things that you already knew about SSIS

10 Things that you already knew about SSIS

Date post: 17-Nov-2014
Category:
Upload: andreas-bergstedt
View: 335 times
Download: 1 times
Share this document with a friend
Description:
This is the slide deck that I used at SQL Saturday Johannesburg 2014 It was a session about best practice and how to fix and improve on some of the most common mistakes made by ETL Developers
Popular Tags:
21
10 Things that you already knew about SSIS A tribute to stupid TAX. By Andreas Bergstedt
Transcript
  • 1. 10 Things that you alreadyknew about SSISA tribute to stupid TAX. By Andreas Bergstedt

2. About Me 20 Years in the IT Industry from Counter Tech to BI Architect, Active speaker at theSQL User groups. Strong focus on practical BI solutions and logical approach tosolving business problems. If it can't be automated it is not worthwhile doing... Twitter:@AndreasBergstedt E-Mail: [email protected] / [email protected] Skype: o0EX-TC0o (Yes I know it is not a professionally looking Skype name) 3. Session Summary About Me and this slide 3 Minutes Who pays stupid tax? 5 Minutes My .dtsx files are corrupted now what? 2 Minutes What is a buffer? 3 Minutes What is Context Switching and why should I care? 3 Minutes My Lookup is so slow Part 1 - 2 Minutes My Lookup is so slow Part 2 - 3 Minutes I dont use a Merge Join because the sort transform is so slow! 5 Minutes I know I can run 32 separate threads because my server has 32 Cores! 2 Minutes Delta what? 10 Minutes What is Parallel processing? 15 Minutes QnA - 7 Minutes 4. Who pays stupid tax? Stupid tax: an additional and generally avoidable taxation upon your regularhabits. 5. Who pays stupid tax? 6. My .dtsx files are corrupted now what? How to fix this: Manual editing (Bad Idea) Get the latest copy from a friend (Highly improbable) Refresh the solution from Source Control (What if it was never checked in) Or maybe. 7. Restore the last build from your .ispac Open the BinDevelopment folder Rename the .ispac to .ZIP Open the .zip file and copy the last .dtsx file that is corrupted back to yourSolution replacing the corrupted file. Open the Project again and. Your smiling. 8. What is a buffer? Default Buffer size in SSIS is 10MB Maximum Buffer size is 100MB Default Buffer row count is 10,000 Rows Maximum Buffer row count is .. There is none. Max Data flow task memory usage = # of Input Buffers + # of Output Buffers* Buffer Size 9. How do I change the buffer size? 10. What is Context Switching and why should Icare? A context switch is the computing process of storing and restoring state(context) of a CPU thread so that execution can be resumed from the samepoint at a later time. This enables multiple processes to share a single CPUthread. 11. When does Context Switching occur? 12. My Lookup is so slow Part 1Dont bring in the entire table in the lookup 13. My Lookup is so slow Part 2Unnecessary buffer usage 14. I dont use a Merge Join because the sort transformis so slow! 15. I know I can run 32 separate threads becausemy server has 32 Cores! SSIS Allows maximum of 10 parallelthreads per package (Citation Needed) Chris Adkin talked about this at SQLBITS XII and had some interesting deepperformance testing that suggested a maximum of 4 threads delivered bestparallel performance 16. Delta what? What is Delta ?The incremental difference between acquired and consumed data. Generational Delta:A new version of an existing business key (Common to DimensionalProcessing) Transactional delta:New transactions since last ETL (Common in fact loads) 17. Delta what? Generational Delta: 18. Transactional Delta: 19. What is Parallel processing? The ability to scale out your data flow in multiple threads 20. Demo 21. QnAThank You


Recommended