Showing posts with label Azure. Show all posts
Showing posts with label Azure. Show all posts

Tuesday, July 28, 2009

Why I am excited about the Azure Platform's .Net Service Bus

Basically, the following statement:

"...Full-duplex, connection-oriented peer-to-peer sessions with network-boundary traversal create direct end-to-end connectivity through NAT"

That's awesome... because any developer that has ever tried to create a real-time, Peer-to-Peer application, he/she knows what hell it is to penetrate firewalls specially Symmetric NATs. Click here to learn about NATs

The Service Bus enables secure, loosely-coupled connectivity between services and applications to navigate firewalls or network boundaries, using a variety of communication patterns. The Access Control service provides an enterprise-class mechanism for federated authorization across organizations and protocols.

In the same way that Microsoft® .NET Framework provides higher-level libraries to make developers more productive, .NET Services help developers focus on their application logic rather than deploying and managing their own cloud-based infrastructure."

Azure's .Net Service Bus enables developers to build applications for the cloud and offers a scalable hosted infrastructure for deploying and managing these applications and their data stores combined with a workflow engine and a security infrastructure. The Azure Service platform is the new OS for internet scaled applications.

Lower Barriers to Building Composite Applications

Using the Service Bus, an application or service can expose and access endpoints that would otherwise be hard or impossible to reach. For example, the endpoints may be located behind network address translation (NAT) boundaries or bound to frequently changing, dynamically assigned IP addresses.
  • Exposes Service Endpoints Easily

    >> Users can access a global hierarchical namespace that is DNS- and transport- independent

    >> Services can be located through a stable, Internet-accessible URL, irrespective of location.

  • Offers Multiple Connection Options

    >> One-way messaging between sender and listener supports unicast and multicast datagram distribution

    >> Full-duplex connection-oriented sessions between sender and listener support bi-directional communication

    >> Full-duplex, connection-oriented peer-to-peer sessions with network-boundary traversal create direct end-to-end connectivity through NAT

  • Supports Publish and Subscribe for Multicasting

    The simple publish/subscribe model lets multiple publishers and multiple subscribers simultaneously use the service’s topic management and event distribution system



More on Azure .Net Service Bus

According to recent traffic studies, the BitTorrent protocol is now responsible for roughly half of all Internet traffic. That's a lot of sharing of personal photos, self-sung songs, and home videos. Half! Next to text messaging, Instant Messaging applications are the social lifeline for our teenagers these days – so much that the text messaging and IM lingo is starting to become a natural part of the colloquial vocabulary everywhere. Apple's TV, Microsoft's Xbox 360, and Netflix are shaking up the video rental market by delivering streamed or downloadable high-quality video and streams on YouTube have become the new window on the world. Gamers from around the world are meeting in photorealistic virtual online worlds to compete in races, rake in all the gold, or blast their respective Avatars into tiny little pieces.

What does all of that have to do with Web 2.0? Very little. While it's indisputable that the Web provides the glue between many of those experiences, the majority of all Internet traffic and very many of the most interesting Internet applications depend on bi-directional, peer-to-peer connectivity.

These familiar consumer examples have even more interesting counterparts in the business and industrial space. Industrial machinery has ever increasing remote management capabilities that allow complete remote automation, reprogramming, and reconfiguration. Security and environment surveillance systems depend on thousands of widely distributed, remotely controlled cameras and other sensors that sit on street poles, high up on building walls, or somewhere in the middle of a forest. Terrestrial and satellite-based mobile wireless technologies make it possible to provide some form of digital connectivity to almost any place on Earth, but making an array of devices addressable and reachable so that they can be integrated into and controlled by a federated, distributed business solution that can leverage Internet scale and reach remains incredibly difficult.

The primary obstacle to creating pervasive connectivity is that we have run out of IPv4 addresses. There is no mere threat of running out, we're already done. The IPv4 space is practically saturated and it's really only network address translation (NAT) that permits the Internet to grow any further. The shortage is already causing numerous ISPs to move customers behind NATs and not to provide them with public IP address leases any longer. Getting a static public IP address (let alone a range) is getting really difficult. IPv6 holds the promise of making each device (or even every general-purpose computer) uniquely addressable again, but pervasive IPv6 adoption that doesn't require the use of transitional (and constraining) tunneling protocols will still take many years.

The second major obstacle is security. Since the open network is a fairly dangerous place these days and corporate network environments are often und unfortunately not much better, the use of Firewalls has become ubiquitous and almost all incoming traffic is blocked by default on the majority of computers these days. That's great for keeping the bad guys out, but not so great for everything else – especially not for applications requiring bi-directional connectivity between peers.

Since these constraints are obviously well-known and understood there is a range of workarounds. In home networking environments the firewall and NAT issues are often dealt with by selectively allowing applications to open inbound ports on the local and network router firewalls using technologies like UPnP or by opening and forwarding port by ways of manual configuration. Dynamic DNS services help with making particular machines discoverable even if the assigned IP address keeps changing. The problem with those workarounds is that they realistically only ever work for the simplest home networking scenarios and, if they do work, the resulting security threat situation is quite scary. The reality is that the broadly deployed Internet infrastructure is optimized for the Web: clients make outbound requests, publicly discoverable and reachable servers respond.

If your application requires bi-directional connectivity you effectively have two choices: Either you bet on the available workarounds and live with the consequences (as BitTorrent does) or you build and operate some form of Relay service for your application. A Relay service accepts and maintains connections from firewalled and/or NAT-ed clients and routes messages between them. Practically all chat, instant messaging, video conferencing, VoIP, and multiplayer gaming applications and many other popular Internet applications depend on some form of Relay service.

The challenge with Relay services is that they are incredibly hard to build in a fashion that they can provide Internet scale where they need to route between thousands or even millions of connections as the large Instant Messaging networks do. And once you have a Relay that can support such scale it is incredibly expensive to operate. So expensive in fact that the required investments and the resulting operational costs are entirely out of reach for the vast majority of software companies. The connectivity challenge is a real innovation blocker and represents a significant entry barrier.

The good news is that Microsoft .NET Service Bus provides a range of bidirectional, peer-to-peer connectivity options including relayed communication. You don't have to build your own or run your own; you can use this Building Block instead. The .NET Service Bus covers four logical feature areas: Naming, Registry, Connectivity, and Eventing.

Naming

The Internet's Domain Name System (DNS) is a naming system primarily optimized for assigning names and roles to hosts. The registration records either provide a simple association of names and IP addresses or a more granular association of particular protocol roles (such as identifying domain's mail server) with an IP address. In either case, the resolution of the DNS model occurs at the IP address level and that is very coarse grained. Since it is IP address centric, a DNS registration requires a public IP address. Systems behind NAT can't play. Even though Dynamic DNS services can provide names to systems that do have a public IP address, relying on DNS means for most ISP customers that the entire business site or home is identified by a single DNS host entry with dozens or hundreds of hosts sitting behind the NAT device.

If you want to uniquely name individual hosts behind NATs, differentiate between individual services on hosts, or want to name services based on host-independent criteria such as the name of a user or tenant, the DNS system isn't an ideal fit.

The .NET Service Bus Naming system is a forest of (theoretically) infinite-depth, federated naming trees. The Naming system maintains an independent naming tree for each tenant's solution scope and it's up to the application how it wants to shape its tree. 'Solution' is a broad term in this context meant to describe a .NET Service Bus tenant – on the customer side, a Service Bus application scope may map to dozens of different on-site applications and hundreds of application instances.

Any path through the naming tree has a projection that directly maps to a URI.

Let's construct an example to illustrate this: You design a logistics system for a trucking company where you need to route information to service instances at particular sites. The application scope is owned by your client, 'ContosoTrucks' which has a number of logistics centers where they want to deploy the application. Your application is called 'Shipping' and the endpoints through which the shipping orders are received at the individual sites are named 'OrderManagement'. The canonical URI projection of the mapping of New York's order management application endpoint instance into the ServiceBus Naming system is
http://servicebus.windows.net/services/contoso/NewYork/Shipping/OrderManagement/

The significant difference from DNS naming is that the identification of services and endpoints moves from the host portion of the URI to the path portion and becomes entirely host-agnostic. The DNS name identifies the scope and the entry point for accessing the naming tree. That also means that the path portion of the URI represent a potentially broadly distributed federation of services in the Naming service, while the path portion of a 'normal' URI typically designates a collocated set of resources.

There is no immediate access API for the Naming system itself. Instead, access to the Naming system is provided through the overlaid Service Registry.

Service Registry

The Service Registry allows publishing service endpoint references (URIs or WS-Addressing EPRs) into the Naming system and to discover services that have been registered.

The primary access mechanism for the Service Registry is based on the Atom Publishing Protocol (APP) allowing clients to publish URIs or EPRs by sending a simple HTTP PUT request with an Atom 1.0 'item' to any name in the naming tree. It's removed by sending an HTTP DELETE request to the same name. There is no need to explicitly manage names – names are automatically created and deleted as you create or delete service registry entries.

Service discovery is done by navigating the naming hierarchy, which is accessible through a nested tree of Atom 1.0 feeds whose master-feed is located at http://servicebus.windows.net/services/[solution]/. Any publicly registered service is accessible through the feed at the respective location.

In addition to the Atom Publishing Protocol, the Service Registry also supports publishing, accessing, and removing endpoint references using WS-Transfer and the Relay service will automatically manage its endpoints in the Service Registry without requiring any additional steps.

The Service Registry is an area that will see quite significant further additions over the next few milestones including support for service categorization, search across the hierarchy, and support for additional high-fidelity discovery protocols.

Connectivity

The core of the connectivity feature area of the .NET Service Bus is a scalable, general-purpose Relay service. The Relay's communication fabric supports unicast and multicast datagram distribution, connection-oriented bi-directional socket communication and request-response messaging.

Towards listening services the Relay takes on the same role as operating-system provided listeners such as Windows' HTTP.SYS. Instead of listening for HTTP requests locally, a relayed HTTP service establishes an HTTP listener endpoint inside the cloud-based Relay and clients send requests to that cloud-based listener from where they are forwarded to the listening service.

The connection between the listener and the Relay is always initiated from the listener side. In most connection modes (there are some exceptions that we'll get to) the listener initiates a secured outbound TCP socket connection into the Relay, authenticates, and then tells the Relay at which place in the naming tree it wants to start listening and what type of listener should be established.

Since a number of tightly managed networking environments block outbound socket connections and only permit outbound HTTP traffic, the socket based listeners are complemented by an HTTP-based multiplexing polling mechanism that builds on a cloud-based message buffer. In the PDC release the HTTP-based listeners only support the unicast and multicast datagram communication, but bidirectional connectivity is quite easily achievable by pairing two unicast connections with mutually reversed client and listener roles.

A special variation of the bi-directional socket communication mode is 'Direct Connect'. The 'Direct Connect' NAT traversal technology is capable of negotiating direct end-to-end socket connections between arbitrary endpoints even if both endpoints are located behind NAT devices and Firewalls. Using Direct Connect you can start connections through the Relay and 'Direct Connect' will negotiate the most direct possible connectivity route between the two parties and once the route is established the connection will be upgraded to the direct connection – without information loss.

With these connectivity options, the Relay can provide public, bi-directional connectivity to mostly any service irrespective of whether the hosting machine is located behind a NAT or whether the Firewalls layered up towards the public network don't allow inbound traffic. The automatic mapping into the Naming system means that the service also gains a public address and the service can, on demand, be automatically published into the Service Registry to make the service discoverable.

In addition to providing NAT and Firewall traversal and discoverability the delegation of the public network endpoint into the Relay provides a service with a number of additional key advantages that are beneficial even if NAT traversal or discoverability are not a problem you need to solve:
  • The Relay functions as a "demilitarized zone" that is isolated from the service's environment and takes on all external network traffic, filtering out unwanted traffic.

  • The Relay anonymizes the listener and therefore effectively hides all details about the network location of the listener thus reducing the potential attack surface of the listening service to a minimum.

  • The Relay is integrated with the Access Control Service and can require clients to authenticate and be authorized at the Relay before they can connect through to the listening service. This authorization gate is enabled by default for all connections and can be selectively turned off if the application wants to perform its own authentication and authorization.

These points are important to consider in case you are worried about the fact that the Relay service provides Firewall traversal. Firewalls are a means to prevent undesired foreign access to networked resources – the Relay provides a very similar function but does so on an endpoint-by-endpoint basis and provides an authentication and authorization mechanism on the network path as well.

If your applications are already built on the .NET Framework and your services are built using the Windows Communication Foundation (WCF) it's often just a matter of changing your application's configuration settings to have your services listen on the Relay instead on the local machine.

The Microsoft.ServiceBus client framework provides a set of WCF bindings that are very closely aligned with the WCF bindings available in the .NET Framework 3.5. If you are using the NetTcpBinding in your application you switch to the NetTcpRelayBinding, the BasicHttpBinding maps to the BasicHttpRelayBinding, and the WebHttpBinding has its equivalent in the WebHttpRelayBinding. The key difference between the standards WCF bindings and their Relay counterparts is that they establish a listener in the cloud instead of listening locally.

All WS-Security and WS-ReliableMessaging scenarios that are supported by the standard bindings are fully supported through the Relay. Transport-level message protection using HTTPS or SSL-protected TCP connections is supported as well.

If the listener chooses to rely on WS-Security to perform its own authentication and authorization instead of using the security gate built into the Relay, the HTTP-based Relay bindings' policy projection is indeed identical to their respective standard binding counterparts which means that client components can readily use the standard .NET Framework 3.5 bindings (and other WS-* stacks such as Sun Microsystems' Metro Extensions for the Java JAX-WS framework).

If you prefer RESTful services over SOAP services, you can build them on the WebHttpRelayBinding using the WCF Web programming model introduced in the .NET Framework 3.5. The Relay knows how to route SOAP 1.1, SOAP 1.2 messages and arbitrary HTTP requests transparently.

The NetEventRelayBinding doesn't have an exact counterpart in the standard bindings. This binding provides access to the multicast publish/subscribe capability in the Relay. Using this binding, clients act as event publishers and listeners act as subscribers. An event-topic is represented by an agreed-upon name in the naming system. There can be any number of publishers and any number of subscribers that use the respective named rendezvous point in the Relay. Listeners can subscribe independent of whether a publisher currently maintains an open connection and publishers can publish messages irrespective of how many listeners are currently active – including zero. The result is a very easy to use lightweight one-way publish/subscribe event distribution mechanism that doesn't require any particular setup or management.

The discussion of the close alignment between the Relay's .NET programming experience and the standard .NET Framework shouldn't imply that the Relay requires the use of the .NET Framework. Microsoft is working with community partners to provide immediate and native Relay support for the Java and Ruby platforms of which initial releases will be available at or shortly after PDC with more language and platform support lined up in the pipeline.

The Relay provides connectivity options that allow you build bidirectional communication links for peer-to-peer communication, allows making select endpoints securely and publicly reachable without having to open up the Firewall floodgates, and provides a cloud-based pub/sub event bus that permits your application to distribute events at Internet scale. I could start enumerating scenarios at this point, but it seems like a safe bet that you can already think of some.

Find out more here:
http://www.microsoft.com/azure/default.mspx
http://www.microsoft.com/azure/servicebus.mspx

Wednesday, March 18, 2009

Azure™ Services Platform

This morning I learned about Windows Azure services platform. Our company had a guy from Microsoft do a presentation. Very interesting and inspiring. I had heard about Cloud computing on several blogs before, but truly realized its power in this session.

Being an entrepreneur, I really think Azure (and other Cloud computing platforms) levels the playing field for Entrepreneurs. Instead of worrying about servers, databases, hosting etc, we can now focus on the core idea and use the "cloud" as the publishing platform. It's awesome, specially in the times we are in, economically speaking!  


"Cloud computing will change how we do IT, end-to-end, over the next five years," says Chuck Hollis, vice president of global marketing at EMC, the Hopkinton-based storage firm.

Before we talk about Azure, let's talk about Cloud Computing.

According to Wikipedia:

Cloud computing is Internet ("cloud") based development and use of computer technology ("computing"). It is a style of computing in which dynamically scalable and often virtualised resources are provided as a service over the Internet. Users need not have knowledge of, expertise in, or control over the technology infrastructure "in the cloud" that supports them. 

The concept incorporates infrastructure as a service (IaaS), platform as a service (PaaS) and software as a service (SaaS) as well as Web 2.0 and other recent technology trends which have the common theme of reliance on the Internet for satisfying the computing needs of the users.

Cloud Computing explained by Canon O' Brian's Twin Brother:


The cloud computing space is now really heating up with competing platform plays from Google’s AppEngine and Amazon’s recent EC2 announcements. In some sense this is perhaps the re-birth of Windows as a platform that enables developers and the ecosystem to build on a whole new set of capabilities and scale for the Software+Services world, in much the same way as Windows fueled the desktop software industry in the past "era". 

The really compelling thing about this particular development platform especially combined with other technologies such as Mesh, SQL and Silverlight is that you’re likely already familiar with it - if you’ve been developing ASP.NET and .NET applications using Visual Studio, either on the desktop or your own servers or hosted servers, you’re already got a good head start in working against this new platform.


What is the Azure Services Platform?

The Azure™ Services Platform (Azure) is an internet-scale cloud services platform hosted in Microsoft data centers, which provides an operating system and a set of developer services that can be used individually or together. Azure’s flexible and interoperable platform can be used to build new applications to run from the cloud or enhance existing applications with cloud-based capabilities. Its open architecture gives developers the choice to build web applications, applications running on connected devices, PCs, servers, or hybrid solutions offering the best of online and on-premises.

Azure reduces the need for up-front technology purchases, and it enables developers to quickly and easily create applications running in the cloud by using their existing skills with the Microsoft Visual Studio development environment and the Microsoft .NET Framework. In addition to managed code languages supported by .NET, Azure will support more programming languages and development environments in the near future. Azure simplifies maintaining and operating applications by providing on-demand compute and storage to host, scale, and manage web and connected applications. Infrastructure management is automated with a platform that is designed for high availability and dynamic scaling to match usage needs with the option of a pay-as-you-go pricing model. Azure provides an open, standards-based and interoperable environment with support for multiple internet protocols, including HTTP, REST, SOAP, and XML.

Microsoft also offers cloud applications ready for consumption by customers such as Windows Live™, Microsoft Dynamics™, and other Microsoft Online Services for business such as Microsoft Exchange Online and SharePoint® Online. The Azure Services Platform lets developers provide their own unique customer offerings by offering the foundational components of compute, storage, and building block services to author and compose applications in the cloud. 




Write Applications to Run On Windows Azure

Developers can start by writing applications to Windows Azure™ by using the Microsoft® .NET Framework and Microsoft Visual Studio®. Write web or mobile applications or author web services. In the future there will be support for both Microsoft and non-Microsoft programming languages and development environments.


Once you’re done coding the application, deploy it to the cloud and run it in Windows Azure and make it available via the internet to your end users. Scale compute capacity up or down based on traffic.



Use Azure Services In Online and On-Premises Applications

Take your cloud application to the next level by adding new functionality using additional Azure services. Use Live Services to reach over 460 million Live users, Microsoft .NET Services for workflow, access control, or service bus functionality, or use the Microsoft SQL Services cloud database. Developers can also write applications and web services that can be consumed by business partners or consumers.

Additionally, Azure services can also be used to augment an existing application that runs on a PC or a server to give on-premises software cloud capabilities. The services use industry standard SOAP, REST and XML protocols so using them won’t be a problem regardless of the operating system or programming language you’re using.


Bring It All Together


The Azure Services Platform is a cloud operating system and collection of services that can deliver web, mobile, or hybrid software-plus-services applications to users. Existing software can utilize the services to add cloud capabilities, and developers can easily write applications for the cloud to be used by end users, or write services that can be consumed within other applications.

Download the Azure Services Platform training Kit (February update)

The Azure Services Training Kit February update is now available and includes the following content covering Windows Azure, .NET Services, SQL Services, and Live Services:

13 hands-on labs – This includes 3 new hands-on labs for Live Framework & Services in addition to the hands-on labs released in the previous version of the kit.

19 new demo scripts – The demo scripts are designed to provide detailed walkthroughs of key services and features so that you can easily present a demo and better understand the services.

10 new presentations – These PowerPoint presentations can be used to deliver presentations at user groups or other training events.


Download the SDKs to experiment with the Azure locally