How to Stay Fresh

After 30 years in the business someone asked me... "How do you stay fresh?".

I had to stop and think about it for a minute and then the answer came to me.

A day or two ago someone pointed me to the samples directory that is included with OBIEE's Paint demo - the basic software that you get from Oracle - to review the entry for a basic time dimension. I opened it up and reviewed it end-to-end. Why? Because I found it to be very interesting and I realized that it is the passion that I have for the details that keeps me going. Maybe you feel the same way? It's that "can't get enough of this" and "I love what I do" that keeps me current.

Working at software companies requires a certain "How are we going to do that?" attitude so it forces you to constantly reach out and learn the tools-of-the-trade that are required to build software - C, C++, Java (Assembler in the old days).... Well for the most part anyway these tools are used to build the software itself.

Applications are built using software so the same passion is required to 'stay fresh' whether you are building software itself or building applications. When building OBIEE Apps you must simply learn to use the tools that Oracle provides and combine that with your business skills.

So now Oracle introduces what I consider to be a third level. First is the software itself. Next are the applications you build using the software. And now third, we have pre-built OBIEE applications that require customization of the application - not the software. This takes the "How to stay fresh" question to a new higher level.

When you look at customizing pre-built applications you must dive into the meta-layer and determine what the application developers where thinking when they built it to spec. Where did the spec come from? How was it established? Does it make sense for my client? Does it need to be customized?

And then there is the GAP (GAAP for financials) Analysis for the loads.... Where does it come from? How does it get there? What adapters so I need? How do I use them? What is the Universal Adapter?

So you have a whole new level of 'customization' consulting that really did not exist up until the combination of the tools that Oracle combined - Informatica, DAC and OBIEE Answers and such.

Staying current or staying fresh in this environment requires a whole new level of passion and experience. You must get into the heads of the application designers and the software developers - both - to stay fresh in this arena.

So let's just say that Oracle has raised the bar on the basic passion and competencies required to "Stay Fresh" in this arena.

If someone walks in and tells you they have "Three Years experience with Oracle's Project Management Analytic App" or "You don't need to use the Informatica piece.." or "The DAC is worthless.." - please kindly ask them to step aside and keep asking questions until you find someone who has the passion to "Stay Fresh !"

New Release of Oracle Business Intelligence 7.9.6

With the new release of Oracle BI Applications, Oracle extends its BI solution offering to include Project Analytics and Loyalty Analytics

Oracle recently announced the introduction of Oracle Business Intelligence (BI) Applications Release 7.9.6. The new capabilities in Oracle BI Applications Release 7.9.6 surrounding Project Analytics and Loyalty Analytics will provide an opportunity to help organizations to manage and execute with greater business insights and agility.

Oracle BI Applications are complete, pre-built BI solutions that deliver intuitive, role-based intelligence for everyone in an organization – from front-line employees to senior management – that enable better decisions, actions, and business processes. Based on best practices, these solutions enable organizations to gain greater insight and value from a range of data sources and applications including the Oracle E-Business Suite, Oracle’s PeopleSoft Enterprise, Oracle’s Siebel CRM, Oracle’s JD Edwards EnterpriseOne, and third party systems such as SAP.

Security!... where do I start?

This post is going to discuss things to consider when setting up a basic security strategy in OBIEE, focusing mainly on User Authentication.

Security in OBIEE can be thought of as a 2 step process as a user logs into the system:
1. Authentication – Do you have access?
2. Authorization – How much access do you have?

Authentication
Think of this as just getting your foot in the door. During Authentication, OBIEE is simply determining if a user has access to the tool. Once someone logs into the system from the Logon screen, the Oracle BI Presentation Server (OBIEE web front end), creates two System Variables. One for the Username that was entered (:USER) and one for the Password that was entered (:PASSOWORD). These System Variables are then sent to the Oracle BI Server to check if a User has been set up with that combination of Username and Password. A User can be set up a few different ways…

Security Manager:

The easiest and most basic way to set someone up with a Username and Password is via the Security Manager. Once someone is set up in this manner, their Username (:USER) and Password (:PASSWORD) are stored on the Oracle BI Server. If the Username and Password entered in the Logon Screen match what’s stored on the Oracle BI Server, they are granted access to the tool. This approach should only be taken if just a few people need access to the tool. Adding an entire organization of people in this manner is tedious, time consuming, and difficult to maintain.

External Database:

Setting up a table with a complete set of User Information, including Username and Password, will allow OBIEE to verify security credentials using what’s called an Initialization Block. An Initialization Block is basically a SELECT statement sent to the User Information table to pull back Username and Password. It is then compared to the System Variables sent from the Logon screen. If there is a match in the User Information table, they are given access. Now the User doesn’t need to be entered through the Security Manager, and the User Information table can be maintained from the database. Additionally, a number of other Variables can be populated from your User Information table to be used during Authorization.

Care must be taken when creating the Initialization Block, as the order of the SELECT statement determines what information is applied to each Variable. For example, in the image below, the value of “full_name” will be applied to the Variable “DISPLAYNAME”.

Single Sign-On:

SSO allows Users a single point of access to all of their organization’s systems. This typically requires the use of an SSO product. View this blog entry for more information on SSO. http://oraclebiblog.blogspot.com/search/label/SSO

Once you iron out your approach with this first step in the process… You’re ready to tackle the task of setting up Authorization. I’ll cover this in a future post…

BI Publisher – OC4J Crashing

BI Publisher – OC4J crashing with “Out of Memory Error”

Repeatedly customers get the “Out of Memory Error” when they are running high volume/complex BI Publisher reports (formerly Oracle XML Publisher) on 32bit environment using OC4J. Why you get this error when you try to generate these kind of reports in BI Publisher and how to resolve this issue?

There are multiple ways you can resolve this issue.

1) Login to OC4J server and Increase the OC4J timeout in server properties because sometime it take more time to fetch the data and oc4j has timeout parameter which exceeds timeout limit and your oc4j crashes.

2) Modify/tune the SQL to avoid complex join to fetch the results faster. By doing this it will use less memory and return the data within timeout timeframe. Also since you are running on 32bit environment change the multi-threading to single threading. It will run slower than normal but you may get the results back.

3) In oc4j_cmd by default the memory is set to 256K. In 32bit environment oc4j is allowed to use maximum of 1536K memory out of 4GB which is Windows has a limit on 32bit environment. This is more than enough but sometime if you have reports which has more than 2000 pages with auto bursting reports (rare case which I have experienced). You can change the memory in oc4j to 512K to 1024K or 1536K by using following command:

Take the following steps to change the heap size values for an OC4J instance:
1. Navigate to the Home page for the OC4J instance.
2. Click Administration.
3. If necessary, expand the Properties section of the table by clicking the Expand icon. Then, click the Go to Task icon in the Server Properties row.
4. In the Command Line Options area, change the value in the Maximum heap size and Initial heap size fields.
5. Click Apply.
6. Navigate to the Cluster Topology page, select the OC4J instance that you modified, and click Restart. On the Confirmation page, click Yes.
This changes the size of the heap allocated to the OC4J process (only for that single instance). If your Oracle Application Server topology includes more than one JVM on the same system, The heap changes must be apply to each JVM.

OR go to C:\OracleBI\oc4j_bi\bin folder and modify oc4j.cmd file, save and restart the OC4J service.

Also you can monitor the heap size by logging to OC4J and check under performance tab:

4) If you are running Presentation server on Windows server, the oc4j memory is shared by BI Presentation sever and BI Publisher. You move the presentation server from OC4J to IIS by re-installing the OBIEE on window box. During installation, it will provide an option for you to use OC4J or IIS for BI Presentation service and you should choose IIS. This way it will release the memory which is used by Presentation service. Also In future if oc4j crashes, still Users will be able to login on OBIEE and run Dashboards & Reports and won’t effect them since your presentation services configure on IIS.

Free to Blog

What a great opportunity for me to speak freely on the OBIEE Blog site. Working with the team here at BICG - outside of direct reporting into a software vendor - inspires 'more honest' advice.

I've seen many versions of BI over the last thirty years. In the early years we had to figure out how to join VSAM files with IMS data and produce a report. Of course, this was in that funny era when we made the leap from card based editors to CRTs. Maybe thats going back a bit too far for any relevent blogging.

Anyway, I thought there might be some interest out there in knowing how to get from say Cognos to OBIEE or Hyperion along with - or - conversion to OBIEE. I am big on "How-To's" so I'll take that approach going forward.

One thing I learned to do early on is to write about things in the right order so I'll offer some dialog to the Bloggers out there. What would you like to hear about first? Moving from Hyperion into OBIEE without getting burned - or - moving from Cognos to OBIEE?

BI Publisher and OBIEE, Part 1

BI Publisher originated as a Java application called "XML Publisher" (hence the "xmlp" naming convention) embedded within Oracle's E-Business Suite. XML Publisher has been an integral component within E-Business Suite as a means to deliver highly refined PDF-formatted output of raw transactional data, primarily for hard-copy printing.

In its incarnation as Business Intelligence Publisher (hereinafter "BIP"), it is essentially the same application at heart but with several powerful new features. Like XML Publisher, BIP is a XML transformation engine that applies XSLT templates (created on the fly from user-generated WYSIWYG templates in RTF format) against raw data to produce reports with highly customized visual layouts in a variety of output formats.

XMLP differs from BIP in two significant ways:

First, BIP has much greater flexibility using a variety of template sources, output formats, delivery methods and scheduling options.

Second, and more importantly, BIP is not restricted to E-Business Suite as a data source - in fact it has been expanded to interface with a wide variety of applications, including JDBC-complient databases, Webservices, Hyperion... and OBIEE.

From a high-level pespective, BI Publisher...

• transforms raw data from various sources (JDBC, Webservices, OBIEE, etc)
• according to a layout designed using native functionality of best-of-breed layout tools (Word, Excel, Acrobat, etc)
• into various file formats (HTML, PDF, Flash, CSV, XML, XLS, etc)
• and delivers them via various mechanisms (Email, FTP, HTTP, WebDAV, Fax, etc)
• either on demand or according to a user-defined schedule
  

Any organization making the investment into OBIEE should understand and take advantage of the extensive data formatting & delivery functionality provided by BI Publisher.

This discussion is divided into two parts illustrating BIP's basic functionality within the context of its two native OBIEE interfaces:

1. with BI Presentation Services as a specialized integration source, wherein Answers Requests are treated as a unique data type (Part 1)
2. with BI Server as a JDBC-compliant SQL database, wherein Presentation Tables are treated as simple database tables (Part 2)
   

Assumptions for this discussion:

• BIP was correctly installed and configured onto a demo system during a full OBIEE installation process (full installation & configuration of BIP can be tricky, but is out of scope for this discussion - see Oracle's documentation for more info)
• OBIEE on this demo system is serving the demo "Sample Sales" repository
• MS Office is installed on the client system (screenshots herein represent MS Office 2007 but should adequately demonstrate functionality for previous versions)
  

PRE-REQ: Install "BI Publisher Desktop" (interfaces desktop client apps Excel and Word with BI Publisher)

1. Log in to OBIEE as Administrator
2. Click to "More Products - BI Publisher"
3. Click to "Business Intelligence"
4. Click on "Template Builder" (in Developer Tools) - Save BIPublisherDesktop.exe then run
   

PART 1: Integration with OBIEE Presentation Services (via Answers Requests)

BIP can interface with the OBIEE Presentation Services as a data source. BIP "sees" existing Answers requests and treats them as reports ready for formatting & delivery. In this exercise we will create a simple Answers request, then connect to it from BIP and build a simple template using MS Word, then observe how we can use native functionality in MS Word to modify the resulting output.

CREATE ANSWERS REQUEST

1. Click to Answers
2. Open the "Sample Sales Reduced" Subject Area
3. Create & save new Request titled "BI Publisher Request" in your personal folder with the following dimensions & facts:
   Products . Brand
   Time . Year
   Facts Revenue . Revenue
   

CREATE BI PUBLISHER REPORT

1. Open BI Publisher ("More Products" - "BI Publisher")
2. Click to "Shared Folders" - "Business Intelligence"
3. Click to "Create a New Report"
4. Enter Name = "Answers Request Demo"
5. Click "Create"
   
   
6. Click "Answers Request Demo" - "Edit"
7. Click "Data Model" = "New"
8. Enter Name = "BI Publisher Request"
9. Select Data source = "Oracle BI Answers"
10. Click Flashlight icon
    

    NOTE: Directory structure matches Web Catalog structure - because BIP here is communicating with Presentation Services

11. Click "users" then [your username] then "BI Publisher Request"
12. Click "Save" (green disk icon in upper LH corner of screen)
    

CREATE TEMPLATE USING WORD

1. Open MS Word
2. Click to "Add-Ins"
3. Click "Oracle BI Publisher"
4. Click "Log on" - Pop up window will appear
   NOTE: Value for Report Server should be the local OBIEE install
   
   
   
5. Click "Login"
6. "Open Template" window will open
7. In "Workspace" drop-down, select "BI Publisher"
8. Expand "Shared Folders" then click "Business Intelligence"
9. In the "Reports" area, click on the "Answers Request Demo" (which you created in BIP above)
   

   NOTE: Directory structure matches BIP structure - because Word here is communicating with BI Publisher

10. In the "Layout Templates" area, click on "New"
11. Click on "Open Layout Template"
    
    
    
12. Click to "Add-Ins"
13. Select "Insert" - "Table wizard"
    
    
    
14. Select report format = "Table" and click "Next"
15. Click "Next" (Data Set = "ROWSET/ROW")
16. Click >> (selects all available fields), then "Finish"
17. Select "Preview" - "PDF"
    
    
    
18. Obey prompt to save document in RTF format
19. PDF viewer will render output
    
    
    
20. Use "Add-ins" - "Insert" to create new objects, then experiment with native Word formatting functionality to create your desired report layout
21. When ready to proceed, select "Add-ins" - "Oracle BI Publisher" - "Upload Template As..."
    
    
    
22. Enter Template Name = "Answers Request Demo Template"

REVIEW NEW TEMPLATE CONFIGURATION IN BIP

1. Return to BI Publisher
2. Click to "Shared Folders" - "Business Intelligence"
3. Click "Answers Request Demo" - "Edit"
4. Click "Report" - "Layouts" - "Answers Request Demo Template"
   

   NOTE: This is the RTF template which you created in MS Word above

5. Click "View" (link in upper RH corner)
   
   
   
6. Voila!
   

PART 2 Integration with OBIEE Server (via Presentation Tables)

(To be continued...)

Oracle BI High Availability: Part 3

The following is the third installment on the topic of High Availability within an OBIEE environment. As I mentioned in previous posts, much of what I’ll be discussing was covered in an Oracle eSeminar which I recently viewed on the subject. To quickly summarize, our basic goal within an HA implementation is to provide multiple instances of all components from the BI Server all the way to the end user, so if anything fails, we have another instance of the same component ready to go. The first two posts on the subject can be found in the blog archives if you’d like to rewind. This installment will feature the HA connections between the Presentation Servers and BI Scheduler Servers as well as between the Presentation Servers and BI Servers. We’ll also look closer at the BI Scheduler Cluster configuration.

First we’ll look at the connection between the BI Servers and the Presentation Servers. This again is handled by the Cluster Controller. In a Windows environment you’ll need to go into the Administrative Tools on each Presentation Server and set up the clustered ODBC data source. To do so, simply ensure that “Is this a clustered DSN?” is checked, then specify the primary and secondary cluster controllers and ports. On a Unix/Linux box, you’ll need to make the following changes to the odbc.ini file:

IsClusteredDSN=Yes

PrimaryCCS=<PrimaryCCS>

PrimaryCCSPort=9706

SecondaryCCS=<SecondaryCCS>

SecondaryCCSPort=9706

One important note to keep in mind is that if you have any clients from which you want to access your repository in online mode, you’ll need the clustered DSN set up on these as well. They’ll need the same connection to the BI Servers as the Presentation Servers will.

The BI Scheduler Cluster Controller assigns the active Scheduler server. In an HA environment, you would have two cluster controllers, a primary and a secondary, in an active/passive relationship. The secondary server will not be used unless the primary is unavailable. The client Controller ports are specified in the respective NQClusterConfig.INI files. The Scheduler configuration will be handled by the Cluster Controllers, so all we need to do in the instanceconfig.xml file is point to the Cluster Controllers, as shown below:

<Alerts>

<ScheduleServer>

ccsPrimary=”<Primary Cluster Controller>”

ccsPrimaryPort=”<Client Controller Port>”

ccsSecondary=”<Secondary Cluster Controller>”

ccsSecondaryPort=”<Client Controller Port>”

</Alerts>

</ScheduleServer>

The other task you would need to complete is to add the BI Scheduler Administrator credentials to the credential store of each ps. The quickest and easiest way would probably be to copy the credential store file from one instance to all other presentation servers.

A few notes about what will occur when the Active Scheduler fails. The transition from one server to another is seamless to the user’s perspective. The users won’t receive any errors, the Cluster Controller will simply detect the failure on the active server and point to the secondary. Any jobs which didn’t complete will be picked up where they left off. One important note to remember is that once the primary server is back up, it will not automatically resume the primary role. Only after the services have been restarted will the primary Scheduler resume its proper role. If any Java, command line, or script jobs are being run during an interruption, they will be restarted when another server is activated and given a new job ID. Take a look at the diagram of the basic HA architecture for the Scheduler Servers.

To configure the Cluster Controller to talk with multiple Schedulers, you must make the following entry in the NQSClusterConfig.INI file on each Cluster Controller:

SCHEDULERS = "scheduler1:9705:9708", "scheduler2:9705:9708";

The first port number for each scheduler will be the rpc port, where the Scheduler will be listening for connections from the Scheduler. The second will be the monitor port, which simply listens for a “heartbeat” from the Scheduler, to confirm it’s still available. This is how the Cluster Controller determines that the primary server has gone down and it needs to look to the secondary server. The default ports for the rpc and monitor ports are 9705 and 9708, respectively.

Configuration of the BI Scheduler Servers themselves can be completed through the Job Manager or via command line using schconfig. All configuration settings are saved in the instanceconfig.xml file of the Scheduler folder, not to be confused with the file of the same name in the web\config folder. The Scheduler can be configured to talk with multiple Presentation Servers and Java Hosts, which will be necessary if you are setting up a true HA environment. If you are using the Scheduler to run script files, you must place them in a shared network file, so that multiple Scheduler Servers can access them. All Schedulers should have read/write access to these files.

That’s going to wrap it up for this post. I’ll finish things up next time with a closer look at the BI Server cluster and how it’s integrated into the HA environment.