website: document how to write providers using the new helper/schema

plugin/server.go

@@ -30,9 +30,11 @@ const MagicCookieValue = "d602bf8f470bc67ca7faa0386276bbdd4330efaf76d1a219cb4d69
 func Serve(svc interface{}) error {
 	// First check the cookie
 	if os.Getenv(MagicCookieKey) != MagicCookieValue {
-		return errors.New(
+		fmt.Fprintf(os.Stderr,
-			"Please do not execute plugins directly. " +
+			"This binary is a Terraform plugin. These are not meant to be\n" +
-				"Terraform will execute these for you.")
+			"executed directly. Please execute `terraform`, which will load\n" +
+			"any plugins automatically.\n")
+		os.Exit(1)
 	}
 
 	// Create the server to serve our interface

website/source/docs/plugins/basics.html.md

@@ -0,0 +1,110 @@
+---
+layout: "docs"
+page_title: "Plugin Basics"
+sidebar_current: "docs-plugins-basics"
+---
+
+# Plugin Basics
+
+This page documents the basics of how the plugin system in Terraform
+works, and how to setup a basic development environment for plugin development
+if you're writing a Terraform plugin.
+
+<div class="alert alert-block alert-warning">
+<strong>Advanced topic!</strong> Plugin development is a highly advanced
+topic in Terraform, and is not required knowledge for day-to-day usage.
+If you don't plan on writing any plugins, we recommend not reading
+this section of the documentation.
+</div>
+
+## How it Works
+
+The plugin system for Terraform is based on multi-process RPC. Every
+provider, provisioner, etc. in Terraform is actually a separate compiled
+binary. You can see this when you download Terraform: the Terraform package
+contains multiple binaries.
+
+Terraform executes these binaries in a certain way and uses Unix domain
+sockets or network sockets to perform RPC with the plugins.
+
+If you try to execute a plugin directly, an error will be shown:
+
+```
+$ terraform-provider-aws
+This binary is a Terraform plugin. These are not meant to be
+executed directly. Please execute `terraform`, which will load
+any plugins automatically.
+```
+
+The code within the binaries must adhere to certain interfaces.
+The network communication and RPC is handled automatically by higher-level
+Terraform libraries. The exact interface to implement is documented
+in its respective documentation section.
+
+## Installing a Plugin
+
+To install a plugin, put the binary somewhere on your filesystem, then
+configure Terraform to be able to find it. The configuration where plugins
+are defined is `~/.terraformrc` for Unix-like systems and
+`%APPDATA%/terraform.rc` for Windows.
+
+An example that configures a new provider is shown below:
+
+```
+providers {
+	privatecloud = "/path/to/privatecloud"
+}
+```
+
+The key `privatecloud` is the _prefix_ of the resources for that provider.
+For example, if there is `privatecloud_instance` resource, then the above
+configuration would work. The value is the name of the executable. This
+can be a full path. If it isn't a full path, the executable will be looked
+up on the `PATH`.
+
+## Developing a Plugin
+
+Developing a plugin is simple. The only knowledge necessary to write
+a plugin is basic command-line skills and basic knowledge of the
+[Go programming language](http://golang.org).
+
+<div class="alert alert-block alert-info">
+<strong>Note:</strong> A common pitfall is not properly setting up a
+<code>$GOPATH</code>. This can lead to strange errors. You can read more about
+this <a href="https://golang.org/doc/code.html">here</a> to familiarize
+yourself.
+</div>
+
+Create a new Go project somewhere in your `$GOPATH`. If you're a
+GitHub user, we recommend creating the project in the directory
+`$GOPATH/src/github.com/USERNAME/terraform-NAME`, where `USERNAME`
+is your GitHub username and `NAME` is the name of the plugin you're
+developing. This structure is what Go expects and simplifies things down
+the road.
+
+With the directory made, create a `main.go` file. This project will
+be a binary so the package is "main":
+
+```
+package main
+
+import (
+	"github.com/hashicorp/terraform/plugin"
+)
+
+func main() {
+	plugin.Serve(new(MyPlugin))
+}
+```
+
+And that's basically it! You'll have to change the argument given to
+`plugin.Serve` to be your actual plugin, but that is the only change
+you'll have to make. The argument should be a structure implementing
+one of the plugin interfaces (depending on what sort of plugin
+you're creating).
+
+While its not strictly necessary, Terraform plugins follow specific
+naming conventions. The format of the plugin binaries are
+`terraform-TYPE-NAME`. For example, `terraform-provider-aws`.
+We recommend you follow this convention to help make it clear what
+your plugin does to users.

website/source/docs/plugins/provider.html.md

@@ -27,15 +27,218 @@ If you don't plan on writing any plugins, we recommend not reading
 this section of the documentation.
 </div>
 
-## Coming Soon!
+If you're interested in provider development, then read on. The remainder
+of this page will assume you're familiar with
+[plugin basics](/docs/plugins/basics.html) and that you already have
+a basic development environment setup.
 
-The documentation for writing custom providers is coming soon. In the
+## Low-Level Interface
-mean time, you can look at how our
-[built-in providers are written](https://github.com/hashicorp/terraform/tree/master/builtin).
-We recommend copying as much as possible from our providers when working
-on yours.
 
-We're also rapidly working on improving the high-level helpers for
+The interface you must implement for providers is
-writing providers. We expect that writing providers will become much
+[ResourceProvider](https://github.com/hashicorp/terraform/blob/master/terraform/resource_provider.go).
-easier very shortly, and acknowledge that writing them now is not the
+
-easiest thing to do.
+This interface is extremely low level, however, and we don't recommend
+you implement it directly. Implementing the interface directly is error
+prone, complicated, and difficult.
+
+Instead, we've developed some higher level libraries to help you out
+with developing providers. These are the same libraries we use in our
+own core providers.
+
+## helper/schema
+
+The `helper/schema` library is a framework we've built to make creating
+providers extremely easy. This is the same library we use to build most
+of the core providers.
+
+To give you an idea of how productive you can become with this framework:
+we implemented the Google Cloud provider in about 6 hours of coding work.
+This isn't a simple provider, and we did have knowledge of
+the framework beforehand, but it goes to show how expressive the framework
+can be.
+
+The GoDoc for `helper/schema` can be
+[found here](http://godoc.org/github.com/hashicorp/terraform/helper/schema).
+This is API-level documentation but will be extremely important
+for you going forward.
+
+## Provider
+
+The first thing to do in your plugin is to create the
+[schema.Provider](http://godoc.org/github.com/hashicorp/terraform/helper/schema#Provider) structure.
+This structure implements the `ResourceProvider` interface. We
+recommend creating this structure in a function to make testing easier
+later. Example:
+
+```
+func Provider() *schema.Provider {
+	return &schema.Provider{
+		...
+	}
+}
+```
+
+Within the `schema.Provider`, you should initialize all the fields. They
+are documented within the godoc, but a brief overview is here as well:
+
+  * `Schema` - This is the configuration schema for the provider itself.
+      You should define any API keys, etc. here. Schemas are covered below.
+
+  * `ResourcesMap` - The map of resources that this provider supports.
+      All keys are resource names and the values are the
+      [schema.Resource](http://godoc.org/github.com/hashicorp/terraform/helper/schema#Resource) structures implementing this resource.
+
+  * `ConfigureFunc` - This function callback is used to configure the
+      provider. This function should do things such as initialize any API
+      clients, validate API keys, etc. The `interface{}` return value of
+      this function is the `meta` parameter that will be passed into all
+      resource [CRUD](http://en.wikipedia.org/wiki/Create,_read,_update_and_delete)
+      functions. In general, the returned value is a configuration structure
+      or a client.
+
+As part of the unit tests, you should call `InternalValidate`. This is used
+to verify the structure of the provider and all of the resources, and reports
+an error if it is invalid. An example test is shown below:
+
+```
+func TestProvider(t *testing.T) {
+	if err := Provider().InternalValidate(); err != nil {
+		t.Fatalf("err: %s", err)
+	}
+}
+```
+
+Having this unit test will catch a lot of beginner mistakes as you build
+your provider.
+
+## Resources
+
+Next, you'll want to create the resources that the provider can manage.
+These resources are put into the `ResourcesMap` field of the provider
+structure. Again, we recommend creating functions to instantiate these.
+An example is shown below.
+
+```
+func resourceComputeAddress() *schema.Resource {
+	return &schema.Resource {
+		...
+	}
+}
+```
+
+Resources are described using the
+[schema.Resource](http://godoc.org/github.com/hashicorp/terraform/helper/schema#Resource)
+structure. This structure has the following fields:
+
+  * `Schema` - The configuration schema for this resource. Schemas are
+      covered in more detail below.
+
+  * `Create`, `Read`, `Update`, and `Delete` - These are the callback
+      functions that implement CRUD operations for the resource. The only
+      optional field is `Update`. If your resource doesn't support update, then
+      you may keep that field nil.
+
+The CRUD operations in more detail, along with their contracts:
+
+  * `Create` - This is called to create a new instance of the resource.
+      Terraform guarantees that an existing ID is not set on the resource
+      data. That is, you're working with a new resource.
+
+  * `Read` - This is called to resync the local state with the remote state.
+      Terraform guarantees that an existing ID will be set. This ID should be
+      used to look up the resource. Any remote data should be updated into
+      the local data. **No changes to the remote resource are to be made.**
+
+  * `Update` - This is called to update properties of an existing resource.
+      Terraform guarantees that an existing ID will be set. Additionally,
+      the only changed attributes are guaranteed to be those that support
+      update, as specified by the schema. Be careful to read about partial
+      states below.
+
+  * `Delete` - This is called to delete the resource. Terraform guarantees
+      an existing ID will be set.
+
+## Schemas
+
+Both providers and resources require a schema to be specified. The schema
+is used to define the structure of the configuration, the types, etc. It is
+very important to get correct.
+
+In both provider and resource, the schema is a `map[string]*schema.Schema`.
+The key of this map is the configuration key, and the value is a schema for
+the value of that key.
+
+Schemas are incredibly powerful, so this documentation page won't attempt
+to cover the full power of them. Instead, the API docs should be referenced
+which cover all available settings.
+
+We recommend viewing schemas of existing or similar providers to learn
+best practices. A good starting place is the
+[core Terraform providers](https://github.com/hashicorp/terraform/tree/master/builtin/providers).
+
+## Resource Data
+
+The parameter to provider configuration as well as all the CRUD operations
+on a resource is a
+[schema.ResourceData](http://godoc.org/github.com/hashicorp/terraform/helper/schema#ResourceData).
+This structure is used to query configurations as well as to set information
+about the resource such as it's ID, connection information, and computed
+attributes.
+
+The API documentation covers ResourceData well, as well as the core providers
+in Terraform.
+
+**Partial state** deserves a special mention. Occasionally in Terraform, create or
+update operations are not atomic; they can fail halfway through. As an example,
+when creating an AWS security group, creating the group may succeed,
+but creating all the initial rules may fail. In this case, it is incredibly
+important that Terraform record the correct _partial state_ so that a
+subsequent `terraform apply` fixes this resource.
+
+Most of the time, partial state is not required. When it is, it must be
+specifically enabled. An example is shown below:
+
+<pre class="prettyprint">
+func resourceUpdate(d *schema.ResourceData, meta interface{}) error {
+	// Enable partial state mode
+	d.Partial(true)
+
+	if d.HasChange("tags") {
+		// If an error occurs, return with an error,
+		// we didn't finish updating
+		if err := updateTags(d, meta); err != nil {
+			return err
+		}
+
+		d.SetPartial("tags")
+	}
+
+	if d.HashChange("name") {
+		if err := updateName(d, meta); err != nil {
+			return err
+		}
+
+		d.SetPartial("name")
+	}
+
+	// We succeeded, disable partial mode
+	d.Partial(false)
+
+	return nil
+}
+</pre>
+
+In the example above, it is possible that setting the `tags` succeeds,
+but setting the `name` fails. In this scenario, we want to make sure
+that only the state of the `tags` is updated. To do this the
+`Partial` and `SetPartial` functions are used.
+
+`Partial` toggles partial-state mode. When disabled, all changes are merged
+into the state upon result of the operation. When enabled, only changes
+enabled with `SetPartial` are merged in.
+
+`SetPartial` tells Terraform what state changes to adopt upon completion
+of an operation. You should call `SetPartial` with every key that is safe
+to merge into the state. The parameter to `SetPartial` is a prefix, so
+if you have a nested structure and want to accept the whole thing,
+you can just specify the prefix.

website/source/layouts/docs.erb

@@ -134,6 +134,10 @@
 				<li<%= sidebar_current("docs-plugins") %>>
 				<a href="/docs/plugins/index.html">Plugins</a>
 				<ul class="nav">
+					<li<%= sidebar_current("docs-plugins-basics") %>>
+					<a href="/docs/plugins/basics.html">Basics</a>
+					</li>
+
 					<li<%= sidebar_current("docs-plugins-provider") %>>
 					<a href="/docs/plugins/provider.html">Provider</a>
 					</li>