Search in the future will look nothing like today’s simple search engine interfaces. If in 10 years we are still using a rectangular box and a list of results, I should be fired.
—Susan Dumais, Microsoft Research, The New York Times, Sunday March 11, 2007
Microsoft’s Susan Dumais has been among the leading lights in search research for more than a decade. No one is in a better position to announce the end of Search 1.0—the small box with the often-irrelevant results list—than she. That she is now betting her job on the emergence of a search 2.0 is a leading indicator you can take to the bank. Search will change drastically in the next 10 years. We will be happy with the results, but getting there is going to take imagination, innovation and a relentless determination to experiment with the emerging technologies the global Web is birthing every year.
While we can’t see all of the details of search 2.0 from where we stand today, we can see very clearly the key direction in which we are heading. We are in the process of moving away from 25 years spent working from a data-centered model for search. We are now replacing this with a customer- or user-centered model for search. This means that the data-centered approach, with its preoccupation with the document and its contents, has come up short. It can never be successful because it cannot deliver any intelligence about the task the user is trying to accomplish or the context of the question the user is trying to answer. Today’s research horizons are all about using the growing numbers of clues in the electronic environment to bring the user’s intention into the center of the search interaction.
We see multiple trends taking hold in the market that will drive the emergence of search 2.0. Some of them are coming from new business mandates; some of them are coming from new technology opportunities; and some of them are coming from new behaviors on the part of information consumers. In this article we will be considering first the business imperatives that are increasing the importance of search. Then we will be looking at three areas in which search 2.0 is taking shape and influencing enterprisewide strategies: as a new element of enterprise infrastructure, as a new enabler for emergent customer interaction models, and as the core for new businesses built-from-the-ground-up for the Web and its new service models.
The Business Mandate
The big battlefields of today’s search wars are all about information consumers. Many of the tenacious IT infrastructure problems of the past decade have largely been solved, leading commentators like Nicholas Carr to declare IT irrelevant in the pages of the Harvard Business Review. But with the increasing severity of the search or findability problem, and the increasing economic role search is now playing in online businesses and marketing strategies, suddenly IT is engaged more intimately with the business than ever before; companies are now struggling with solving the information-access puzzle for their customers, and often their customers’ customers.
More than 50% of C-level executives recently surveyed by the Economist Intelligence Unit and FAST indicated that the emergence of ubiquitous broadband is the biggest single technological influence on their business strategy. It is this kind of thinking that is driving a new kind of information sophistication that recognizes that the firm’s online experience is becoming a primary new differentiator. In face of growing data volumes and complexity, it is increasingly search that provides the tools that allow companies to extract the most business value out of their unique strength—their knowledge and experience, as it is captured in their content and their connections and interfaces to their users and customers.
Innovative firms are now beginning to implement search in a manner that extends across the information value chain from intelligent infrastructure, through a suite of connection services, and into the contextualized interaction with customers. They have discovered that search is a primary method to connect customers with answers, analytics, actionable services and other people who are relevant to completing a task or taking a decision. Search is a mechanism to help translate the information universe into terms that customers understand, to help understand what the customer intends to accomplish and, of course, to deliver what the customer wants.
The new architecture of search 2.0 offers enterprises the ability to address three core strategic objectives that will help drive the next wave of business online: consumption-oriented IT architectures; agile service delivery platforms; and extreme business innovation.
Consumption-Oriented Architecture
Good architectural principles and deep understanding of established frameworks have always been critical to successful adoption and integration of new infrastructure capabilities. Search is emerging as the orchestration mechanism for user-centric SOA-based architectures, offering innovative functionality for re-engineering both the user experience and the data access layers that support it.
We find some leading examples of search innovation at the data access layer of some of the largest telecom and financial enterprises in the world. Their experience reveals the following five advantages that are characteristic of a search 2.0 approach:
1. High performance, scalability and reliability, with ability to sustain high throughput, low latency request service and scale in terms of request rates, content volumes and 24x7 reliability.
2. Lowered integration effort, since they deliver data in a single, standardized format and provide access over a common API, and they facilitate easy expansion of application scope and the re-use of data from multiple sources.
3. Easier change management, due to a stable and extensible data format (XML) and a standard access pattern, so that changes in sources or consumers can proceed without impact.
4. Consistent operational model, with a reduction of the complexity of the security model. As a central component, the access layer provides a single point which authenticates, authorizes and records access to information. It simplifies performance monitoring—the access layer and portal instances are built to run in high-volume environments and include operational performance metrics and tools.
5. Simplified rollout model, leveraging several existing processes that are already in progress, relating to corporate information access.
A search-powered consumption-oriented architecture does not involve replacing the current infrastructure. It is about getting a better return on past infrastructure investments, better infrastructure re-use and higher levels of abstraction that create value in delivering new business applications. It also enables real-time capabilities based on a scalable solution on top of the already existing infrastructure. This is an evolutionary approach to acquisition and unlocking of revenue streams through search.
Search 2.0 as New Architecture
After internal analysis, one of world’s largest telcos realized that it had several problems in the information access area: limited knowledge of customers, products and context, extended product development and delivery time, absence of an architectural strategy to deal with changes in the global online marketplace and poor customer experience on existing sites. As a consequence, they were unable to deliver relevant, personalized online experiences, leading to customer frustration and potential loss of sales, while lengthy, expensive time-to-market projects reduced their ability to introduce innovative products or competitive offerings rapidly in response to new market entrants. They decided to solve the problem by introducing a new “Information Access and Aggregation Layer,” an architectural pattern based on search 2.0.
In this pattern, the information access layer abstracts content ingestion, format and access from consumers, while enterprise services expose business entity operations such as creation, update and removal of customer data or product orders. The pattern ensures the following five basic properties of the system: 1. high performance, scalability and reliability; 2. low integration effort; 3. simpler change management; 4. clean operational model; and 5. consistent rollout model. Their new search 2.0-based system also introduces several additional capabilities: advanced matching and relevance, intent determination, personalization, content subscription and more.
The result: a significant competitive advantage based on better information access and flexibility of their highly distributed IT platform.
This enriched high-performance level of data access provides each user with information as it changes or as it develops from streams or transactions developing from multiple sources, offering the customer a complete 360-degree view. No matter how fast the information volume grows, search 2.0 solutions scale to support millions of simultaneous users—submitting thousands of queries per second—searching terabytes of data, while still delivering sub-second responses. Real-time indexing and alerting features in search 2.0 can now provide the foundation for a wide range of user-centered applications.
There are many reasons for enterprises to undertake re-engineering of the underlying information landscape: re-organizations, mergers, compliancy requirements, re-use of information, cutting operating costs or simply a need for a richer picture of customers or situations. As systems grow and their intended use changes over time, the architectural landscape must adjust, but without the consumption-centered architecture of search 2.0, this is slow, costly and error-prone. Modern enterprise search platforms help insure against this effect.
Agile Service Delivery Platforms
In the world of the Web, the evolution from Web 1.0 toward Web 2.0 has seen a global transition from monolithic services, centralized content models and managed communities, to a new democracy of empowered users, personalized information access, and user-driven communities. In this new ecosystem of information, evolution favors services that are mashups of component functionalities, resulting in complete interaction environments that are focused by user intent and customized to specific task or discovery goals.
This Lego-model of information sources and services opens up many new opportunities for service orchestration. Good service DNA now includes the ability to connect to open components to harness innovation, the ability to: track and utilize behavioral information across different applications; integrate components loosely to speed up the innovation cycle; bridge information between independent services; connect user experience to social networks; and provision these services through multiple channels as appropriate to the user. Approaches based on search 2.0 enable many of these capabilities.
In orchestrating these service delivery environments, search 2.0 is no longer a small box in one of many disjointed portals. Search is becoming the new driver for the user experience, joining, integrating and connecting many disparate information sources. Unlike their immediate predecessors, search-powered portal views are flexible and modular, they lower the barriers to information integration and reuse, and they lower the time from conception to the full integration of new services and new business based on these. The traditional portals were information gateways; the new portals based on search 2.0 are interactive, collaboration-oriented multimedia information discovery, delivery and analysis tools.
Search 2.0 in Service Delivery
A major publisher in Scandinavia has recently launched a local directory. From the start they were aware of the high-level competition from several financially strong
companies, and realized early that in order to win this market, they had to be best at managing local presence and being closer to the market, content quality and flexibility in local search development. They decided to compete through a complete set of information services, combining structured and unstructured content and merging all content into one search experience. This search experience was significantly enriched through cleansing and query analysis, and has resulted in a number of improved services, including directory information, TV programs, Wikipedia information, classifieds etc.
In short, their recipe for success includes collecting data from multiple sources, combining structured and unstructured data, moving data between internal sources (database to database, index to database, database to index, etc.), cleansing of data through linguistic analysis, fuzzy matching and approximate matching and exporting of data to partners. They based this strategy on search 2.0 tools and most updated data. The result: the data warehouse award of the year in 2006.
Key differentiators in this new service orchestration are the personalized user experience, with communities, tailored views and individualized discovery processes, atomic service integration, where atoms mix and match and enable variable channel delivery and precision and quality of searches, elevating data to insight and monetizing this insight. In all of these capabilities, search 2.0 is the core platform and enabler for orchestrated Web or enterprise service integration.
Search connects users to other people, to services, to useful knowledge. With search 2.0, it is relevant information that finds the user, not the other way around. The search process is becoming absorbed in the continuous process of the customer’s interaction with his environment, and the borderline between the implicit and explicit information provided by the user is becoming fuzzy. All information traces are relevant and important to the improved user experience. The single
access portal, connecting all point portals, is driven by powerful and flexible search, the framework for personalized user experience. Search is becoming the framework of choice to
support the user’s world of connectedness, the new interaction paradigm across formats, devices and environments.
Business Innovation
One of the cornerstones of a competitive online strategy is the ability to rapidly create, deploy and extract services and applications. This innovation speed is more important now than ever before. Across telecom, media and financial industries, companies face the most intense transformation in their history, due to strong market convergence caused by digitalization of content, growth of online services and entrance of non-traditional competitors. They race to find new business models that will maintain and protect existing customer relationships in a world where competition arises from new directions. The ability to quickly identify, test and launch profitable services in the industry’s converging landscape will determine who will win in the market. Capital investment should pay back, many times over, but only if their effect is targeted and immediate.
Search 2.0 is redefining what it means to shape the firm’s “digital front window,” highlighting the information that businesses want to bring forward, for internal and external users alike. In this sense, search is at the heart of helping operators characterize who they are and what they do. Indeed, there are many examples where search is at the core of new business models and at the base of new operational efficiencies. This list is a sampling of capabilities a robust search technology can enable: federation, merchandizing management; data cleansing; concept identification management; advertising management; taxonomy management; rich media mining; tracking and monitoring; interactive reporting; sentiment analysis; location search; real-time alerting; scope search; cross-device navigation and discovery; user group profiling and personalization; recommendation engines; semantic analysis; and more.
Search 2.0 Builds a Business
One of the world’s leading B2B publishers, with more than 2,000 academic journals and 1,900 new books per year, competes in the digital arena through industry-leading, focused innovative online products, which are released
and updated on a regular and aggressive schedule.
Their competitive strategy is targeted on increasing end-user productivity and addressing information overload. They have achieved these goals through four steps: 1. use of search tools that are able to leverage their unique knowledge of market, content and users; 2. customize their data to include only relevant selections from both publisher-controlled content and open Web sources; 3. provide more accurate and relevant search results to users who are dissatisfied with failed general searches; and 4. expose tailored means for navigating and searching metadata, categories and taxonomies. They have provided highly targeted and fresh services through careful segmentation of their markets based on user needs, and they have carefully sliced and diced their content to differentiate their services at point of discovery and delivery. The resulting systems provide authoritative, selective information enhanced with proprietary metadata and cross-linking, integrated into the user’s workflow and superior tools for refining and analyzing results. The result: several “best search engine” awards, as well as significant revenue growth.
Gartner has identified this “information access technology” as the single most important technology that strings across business to derive relevant insight and create significant value from investments in data. Search enables businesses to thrive in the new digital world, by allowing them to quickly develop new business models and move into new markets. In a world of real-time information requirements, ad hoc access to information and short time to market with new services are a core capability to protect current revenue and drive new sources of revenue.
The three core strategic objectives of online strategy based on search 2.0 do not come risk-free. A rich strategic framework requires a clear business-driven roadmap, synchronized across business functions and executed in an evolutionary acquisition and deployment manner. The search 2.0 platform approach can accelerate acquisition of new capabilities and provide recurring improvements to the system.
Search is the New Portal
“Search is the Portal” is the new mantra to financial statistics aggregators such as Reuters and the Financial Times, which have replaced database-centric access solutions with search for their top-of-the-line trading platforms and premium content on-line offerings. Business publishers, whether Hoovers in the US or
the exceptionally innovative Schibsted in Scandinavia, are pushing the frontiers of how to integrate data from multiple (20-100+) separate databases with radically different data models into a semantically uniform representation where unique objects such as companies, individuals, products are created with the joined and cleansed information from data residing in these previously separate silos.
Analysts now agree that search 2.0 technology is the emerging driving force behind strategic successes of businesses from every market and every region of the world. If the network has become the computer, search is in the process of becoming its interface. Search is now a primary business interaction tool, and its enormous differentiating potential is still there for those who appreciate its full strategic value.
The main challenge for businesses competing in the online space is no longer their ability to gather enormous amounts of information. The challenge for attracting and retaining customers, and thereby innovative revenue channels, now lies in making search demonstrably relevant to the customer’s intent. This is the core issue that drives researchers like Susan Dumais. The fact that she is confident that search 2.0 will emerge in the next decade is based on our increasing ability to answer a few key questions like: “Who is this customer?” and “What is she and her social network interested in?” “Where is she?” “What device is she using?” Search 2.0 is providing the basis for firms to automatically understand the user’s search habits and preferences, and provide an algorithmic opportunity to engage in smarter communication with customers, partners and employees. As renowned knowledge entrepreneur Esther Dyson offered in a recent interview with FAST: “How can you not do a better job of serving someone if you know them?”