Add way for dashboard to check for updates (phone home)

[peterfarrell]

Added way for dashboard to check for updates (phone home) for dashboard and for framework. Use the version number to check against a url at mach-ii.com.

[peterfarrell]

Derrick, assigning this ticket to you since you wanted to work on something like this. It would be great for this feature to not only check to see if there is a new dashboard version (and auto-update) but something similar for the framework itself.

[derrickjackson]

Peter, are you saying there will be a URL on mach-ii.com that I can hit that will have the latest version of the dashboard?

format:  http://mach-ii.com/dashboard/[MAJOR VERSION]/[MINOR VERSION]

[peterfarrell]

Well, we might need to restructure how we publish code. Right now we publish stable / final releases to mach-ii.com/code/ (this include dashboard and framework) and nightly releases at mach-ii.com/nightly with an archive of all the nightlies at mach-ii.com/nightly/older/

So for the dashboard, you'll have to build a system to talk to the website and we'll let you build that part as well - so you can build it in a manner that works. That is why you have repo access to the website so you can build the "server" part.

Let me know how you think we should structure our publishing process. I was thinking there should be an option in the dashboard to auto-update to official releases only (Beta, RC1, RC2, Stable/Fina?) or to optionally auto-update with the nightly (because we are trying to encourage developers to use the nightlies more.

Let's continue the discussion here.

[peterfarrell]

FYI, we publish nightlies via ANT and FTP. I think we'll need a way to let the server know there is a new nightly, so I was thinking there could be a "ping" URL we could hit at the end of the nightly publish (with a password or something) that would let the server know to check for a new version. As for the official releases (Beta, RC, Stable, etc.) that is more manual process. I'm thinking we need to build a start to an admin tool for us on the website -- one feature could be a place to set the latest official release version numbers which would update your phone home system as well as change the version number on the website views (because it's a manual process right now).

[derrickjackson]

Ok... I'm free thinking here so forgive me if i get off track. But here is an idea i have.

1. Allow the dashboard "owner" to set a module property named "autoUpdateType". Here are the potential values: 0=Do Not Auto Update, 1=Stable Release Only, 2=BER/Nightly.

2. When nightlies are published an XML file is created. The XML file can include the following nodes: Title, Major Version, Minor Version, Date, FileName?. This file would only have two elements. The first element would be the current nightly information and the second node would include the previous nightly information.

3. Dashboard would then <cfhttp> to that XML file and read the content.

4. Based on the property set in the module the dashboard could then <cfftp> to the necessary location, get the zip file, then unzip it to the correct location.

Before I go any further, what do you think of the ideas so far? I still need to think through how to notify the user there is a new version (JS popup, message, etc) and how to reinit dashboard and all MachII applications that utilize the dashboard automatically.

Can dashboard run as a standalong application that "manages" all Mach-II applications or does it have to run as part of an existing application?

[derrickjackson]

Hopefully I haven't just over-simplified the issue/task.

[peterfarrell]

No over simplification here Derrick.

1. I like this idea. I've been toying with the idea of having some persistent data store for the dashboard (most likely an xml or .properties file) but for now the Mach-II property is a great place to start.

2. Yea, it could be WDDX as well -- it might be even more simple since you don't have to parse the XML file. Either would be fine with me.

3. Yep.

4. Yep exactly. However, since we're have to be cross CFML engine compatible you won't be able to use cfzip and use the Java libraries out there (java.util.zip package). Should be a nice little diversion into Java for you.

Well, as things are currently setup -- there is one dashboard per application. If you instead the dashboard in a centralized location - then if one application updates the Dashboard then all instance will use that code in the end. I'm thinking that we'll have to create a "version" hash in memory so if the version number changes other dashboards will know to reload themselves. From within the dashboard module, the follow code will reload the dashboard (because we are in the context of the dashboard and each module has it's own AppManager):

getAppManager().reloadModuleConfig()

So this version hash is as easy as checking a file on disk that ships with the dashboard and comparing a string against the hash of the version of the dashboard loaded in memory.

As for notifying the user, AJAX or even a straight up html page refresh is fine. We haven't decided on an AJAX library, but know we want to AJAX-ify things in dashboard. Is JS something you are interested in? I was thinking of using either JQuery or Protoype (the later I'm more familar with). Right now we have pages auto-refreshing by doing a normal get, it would be nice to have AJAX get the new data and when say something "reloads a module or a ColdSpring? factory" that it would do it via AJAX instead of hitting a normal page and then doing a refresh.

[derrickjackson]

Peter,

I pulled down the latest files from SVN and I've started getting this error:

The method getFilterChannels was not found in component /Library/WebServer/Documents?/MachII/logging/filters/GenericChannelFilter.cfc.

For some reason this error seems familiar but I cant seem to recall where I've seen it.

Replying to peterfarrell:

No over simplification here Derrick.

1. I like this idea. I've been toying with the idea of having some persistent data store for the dashboard (most likely an xml or .properties file) but for now the Mach-II property is a great place to start.

2. Yea, it could be WDDX as well -- it might be even more simple since you don't have to parse the XML file. Either would be fine with me.

3. Yep.

4. Yep exactly. However, since we're have to be cross CFML engine compatible you won't be able to use cfzip and use the Java libraries out there (java.util.zip package). Should be a nice little diversion into Java for you.

Well, as things are currently setup -- there is one dashboard per application. If you instead the dashboard in a centralized location - then if one application updates the Dashboard then all instance will use that code in the end. I'm thinking that we'll have to create a "version" hash in memory so if the version number changes other dashboards will know to reload themselves. From within the dashboard module, the follow code will reload the dashboard (because we are in the context of the dashboard and each module has it's own AppManager):

getAppManager().reloadModuleConfig()

So this version hash is as easy as checking a file on disk that ships with the dashboard and comparing a string against the hash of the version of the dashboard loaded in memory.

As for notifying the user, AJAX or even a straight up html page refresh is fine. We haven't decided on an AJAX library, but know we want to AJAX-ify things in dashboard. Is JS something you are interested in? I was thinking of using either JQuery or Protoype (the later I'm more familar with). Right now we have pages auto-refreshing by doing a normal get, it would be nice to have AJAX get the new data and when say something "reloads a module or a ColdSpring? factory" that it would do it via AJAX instead of hitting a normal page and then doing a refresh.

[peterfarrell]

Derrick, just fixed that error this afternoon. Try getting the latest BER from Mach-II 1.6 BER (not RC2).

[derrickjackson]

Hmmm. I got the BER for Mach-II and Dashboard and also restarted CF but I still get the error.

Replying to peterfarrell:

Derrick, just fixed that error this afternoon. Try getting the latest BER from Mach-II 1.6 BER (not RC2).

[peterfarrell]

Replying to derrickjackson:

Hmmm. I got the BER for Mach-II and Dashboard and also restarted CF but I still get the error.

Check the GenericChannelFilter.cfc file and see if the getFilterChannels() method is marked the access of public. It should be public and that is what is fixed in the repo right now. Hmm - I suspect that you might have save java class files on in the CFIDE in the Caching tab? Have you tried clearing the template cache in the Caching tab?

[derrickjackson]

I checked the CFC and you hit the nail on the head. The getFilterChannels() method was set to private. I changed it to public and all worked.

Replying to peterfarrell:

Replying to derrickjackson:

Hmmm. I got the BER for Mach-II and Dashboard and also restarted CF but I still get the error.

Check the GenericChannelFilter.cfc file and see if the getFilterChannels() method is marked the access of public. It should be public and that is what is fixed in the repo right now. Hmm - I suspect that you might have save java class files on in the CFIDE in the Caching tab? Have you tried clearing the template cache in the Caching tab?

[derrickjackson]

Peter,

Do I have posting access to http://www.mach-ii.com/nightly/ and http://www.mach-ii.com/code? I would like to place a .cfm file in each location. This file will produce the wddx data I need to read the versions of dashboard and the framework that are available. Once the file is there I will <cfhttp> to each of them.

I know you said you were open to changing the way files are named, so what do you think of this idea. It might be a bit too structured so totally understand if you don't like it.

http://www.mach-ii.com/code/mach-ii http://www.mach-ii.com/code/dashboard

then under each of the directories you would have a version directory

http://www.mach-ii.com/code/mach-ii/1-0-0 http://www.mach-ii.com/code/mach-ii/1-5-0 http://www.mach-ii.com/code/mach-ii/1-6-0, etc

then under each of the version directories you have the actual release zip files

http://www.mach-ii.com/code/mach-ii/1-6-0/MachII 1-6-0 Beta.zip http://www.mach-ii.com/code/mach-ii/1-6-0/MachII 1-6-0 RC1.zip http://www.mach-ii.com/code/mach-ii/1-6-0/MachII 1-6-0 RC2.zip http://www.mach-ii.com/code/mach-ii/1-6-0/MachII 1-6-0 Stable.zip

I would then take the version the user has and drill down to that version folder to get all available releases. Based on the users configured update method and current version date (will have to determine where to save the versions last update date) I would retrieve the new version (if there is one).

Example: Lets assume my autoUpdateType is set to 2(stable/rc) User Version of Dashboard = MachIIDashboard 1-0-0 RC1 User Version date updated = 01/01/09

Website Version of Dashboard = MachIIDashboard 1-0-0 RC2 Website Version date updated = 02/10/09

Since the releases are diff I will ignore the date updated. I will use <cfftp> to grab the file (MachIIDashboard 1-0-0 RC2.zip), use Java to unzip it (need to determine where to save the file and unzip it), move the unzipped files over to the MachIIDashboard folder, delete the temp unzip location (if necessary), and finally re-init Dashboard and MachII(if necessary).

I hope I didn't ramble on to the point of not making sense. Let me know what you think.

[peterfarrell]

Great Idea Derrick! Maybe instead of .cfm files, we could just add some Mach-II event-handlers to the website application. You could create an auto-update module that can be re-used for thinks like MachBlog? as well by providing some configuration information in the XML configuration file.

So the phone home stuff would hit an URL like (if the module is called update):

http://www.mach-ii.com/index.cfm/go/update:checkIfOutDated/package/framework/version/1.6.0.0/

It would hit that event, passing in the package name (in this case we'll support framework and dashboard) and the current version number they are running. This could just return a zero length string to indicate latest version is running or a URL string with the location of the latest version and then do the stuff you already outlined. This will returned cached data.

Also, this module would have another event we can hit to cause it look at the folders for updated versions of the passed package and update the value of the cached data that the previous event I just mentioned returns.

http://www.mach-ii.com/index.cfm/go/update:newPackage/package/framework/

This URL would be hit by our ANT tasks when publishing new ZIPs of the framework or dashboard. We could even password protect this url if it becomes a problem.

How does that sound to you?

[derrickjackson]

Replying to peterfarrell:

Great Idea Derrick! Maybe instead of .cfm files, we could just add some Mach-II event-handlers to the website application. You could create an auto-update module that can be re-used for thinks like MachBlog? as well by providing some configuration information in the XML configuration file.

So the phone home stuff would hit an URL like (if the module is called update):

http://www.mach-ii.com/index.cfm/go/update:checkIfOutDated/package/framework/version/1.6.0.0/

It would hit that event, passing in the package name (in this case we'll support framework and dashboard) and the current version number they are running. This could just return a zero length string to indicate latest version is running or a URL string with the location of the latest version and then do the stuff you already outlined. This will returned cached data.

Also, this module would have another event we can hit to cause it look at the folders for updated versions of the passed package and update the value of the cached data that the previous event I just mentioned returns.

http://www.mach-ii.com/index.cfm/go/update:newPackage/package/framework/

This URL would be hit by our ANT tasks when publishing new ZIPs of the framework or dashboard. We could even password protect this url if it becomes a problem.

How does that sound to you?

[derrickjackson]

I like that idea. I'm going to start working on the module now. Stay tuned for me details/questions.

[derrickjackson]

Replying to peterfarrell:

I like that idea. I'm going to start working on the module now. Stay tuned for me details/questions.

[derrickjackson]

Peter,

I have a few quesitons:

1. In your comments below, when you say "This will return cached data", what type of cached data is being returned?

2. Are you saying you agree with my proposed directory structure for framework and dashboard locations and you plan to implement this? Then based on that structure I will know where to grab the correct updates for framework or dashboard.

3. I'm not sure I follow this part of your comments: "Also, this module would have another event we can hit to cause it look at the folders for updated versions of the passed package and update the value of the cached data that the previous event I just mentioned returns. "

Replying to peterfarrell:

Great Idea Derrick! Maybe instead of .cfm files, we could just add some Mach-II event-handlers to the website application. You could create an auto-update module that can be re-used for thinks like MachBlog? as well by providing some configuration information in the XML configuration file.

So the phone home stuff would hit an URL like (if the module is called update):

http://www.mach-ii.com/index.cfm/go/update:checkIfOutDated/package/framework/version/1.6.0.0/

It would hit that event, passing in the package name (in this case we'll support framework and dashboard) and the current version number they are running. This could just return a zero length string to indicate latest version is running or a URL string with the location of the latest version and then do the stuff you already outlined. This will returned cached data.

Also, this module would have another event we can hit to cause it look at the folders for updated versions of the passed package and update the value of the cached data that the previous event I just mentioned returns.

http://www.mach-ii.com/index.cfm/go/update:newPackage/package/framework/

This URL would be hit by our ANT tasks when publishing new ZIPs of the framework or dashboard. We could even password protect this url if it becomes a problem.

How does that sound to you?

[peterfarrell]

Replying to derrickjackson:

1. In your comments below, when you say "This will return cached data", what type of cached data is being returned?

Your XML or JSON -- the data you are returning to the client. I think I just mentioned this so we can leverage the caching package to keep the same code from run on requests for many dashboards. Don't worry we'll run the optimization bridge when we get there.

2. Are you saying you agree with my proposed directory structure for framework and dashboard locations and you plan to implement this? Then based on that structure I will know where to grab the correct updates for framework or dashboard.

Yeah, I have no problem restructuring how the code is deployed. Have you looked at how the nightlies are released ( http://mach-ii.com/nightly/)? The latest and greatest are always in the root in the folder and the rest live up higher.

3. I'm not sure I follow this part of your comments: "Also, this module would have another event we can hit to cause it look at the folders for updated versions of the passed package and update the value of the cached data that the previous event I just mentioned returns.

Oh, I was saying it would be nice if you could expose an event that has some authentication on it so the team could hit when a new version is published to clear the cached data you return (so the latest update would become effective). This is more of an administration like event than something we need to expose publicly. This event could just get your code to "rescan" the code directory for new code and update the cache.

[derrickjackson]

Sorry for all the questions, but I'm trying to wrap my head around the work flow for getting new versions.

I'm assuming that there will be a JSON or XML or WDDX file on http://www.mach-ii.com/nightly/ that contains the information I need to check against and when a new nightly/version is published the ANT script or something will hit the administrative URL to clear the current cached JSON or XML file then create another cached version. Am I following what you had in mind or am I off base?

[peterfarrell]

Well, it could be physical file as you suggest, but we have a DB here (MySQL).

I was thinking that our ANT script that publishes versions (whether a release like beta, RC1, stable or nightlies) would hit a secured event-handler with some data (package - framework, dashboard, sample app name and a version number). This would insert into a db and do whatever is necessary to get the db into the right state when a new version is added.

You would just query the db instead of search through a file. Or if you don't want to build out an event we can hit -- you could build a scheduled task that would scan the directories and look for new packages.

You could use the db information to create an XML packet (that the dashboard itself could request) by hitting another event in which the describes the newest available packages.

[anonymous]

Milestone Dashboard 1.1.0 alpha1 deleted

[derrickjackson]

Peter The DB will work PERFECTLY. I'm ironing out the details for downloading the ZIP file and I want to be sure an earlier suggestion I made is still what you have in mind. Here is what I said earlier:

I will use <cfftp> to grab the file, use Java to unzip it (need to determine where to save the file and unzip it), move the unzipped files over to the package folder, delete the temp unzip location (if necessary), and finally re-init the package.

If so maybe we can have some anonymous ftp account that can read only. Thoughts before i proceed.

[peterfarrell]

hmm, maybe just use <cfhttp> saves us the trouble of maintaining an anonymous FTP account and those unfortunate people that have FTP blocked at work. Using <cfhttp> allows everything over port 80 which nobody should have blocked (unless they don't have internet connectivity). Otherwise using Java to unzip it, etc. as you state after that sounds good to me.

[derrickjackson]

Ok... The code to get the available versions is written and has been added to the Dashboard 1.1.0 SVN location. I am working on the code that will need to be executed when the ANT scrit is run. It will update the database and indicate the active version of the dashboard and/or framework.

In the meantime, here is the script needed to create the database with data for current contents of the mach-ii.com/code and mach-ii.com/nightly directories:

CREATE TABLE "activePackages" ( 
	"packageID"    	int(11) AUTO_INCREMENT NOT NULL,
	"packageName"  	varchar(50) NULL,
	"version"      	varchar(20) NULL,
	"releaseType"  	int(11) NULL,
	"packageStatus"	int(11) NULL,
	"fileLocation" 	varchar(100) NULL,
	"dateReleased" 	date NULL,
	"dateUpdated"  	date NULL,
	"build"        	varchar(25) NULL,
	PRIMARY KEY("packageID")
);

INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(1, 'framework', '1.6.1', 1, 1, 'http://www.mach-ii.com/code/MachII_1-6-1.zip', '2009-03-29', '2009-03-29', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(3, 'framework', '1.6.0', 1, 0, 'http://www.mach-ii.com/code/MachII_1-6-0.zip', '2009-02-09', '2009-02-09', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(4, 'framework', '1.6.0', 2, 0, 'http://mach-ii.com/code/MachII_1-6-0_RC2.zip', '2009-01-05', '2009-01-05', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(5, 'framework', '1.6.0', 3, 0, 'http://mach-ii.com/code/MachII_1-6-0_RC1.zip', '2008-11-09', '2008-11-09', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(6, 'framework', '1.6.0', 4, 0, 'http://mach-ii.com/code/machII_1-6-0_Beta.zip', '2008-09-22', '2008-09-22', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(7, 'framework', '1.5.0', 1, 0, 'http://mach-ii.com/MachII_1-5-0.zip', '2008-05-05', '2008-05-05', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(8, 'framework', '1.5.0', 3, 0, 'http://mach-ii.com/MachII_1_5_0_RC1.zip', '2008-04-16', '2008-04-16', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(9, 'framework', '1.5.0', 4, 0, 'http://mach-ii.com/MachII_1_5_0_Beta1.zip', '2008-05-05', '2008-05-05', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(10, 'dashboard', '1.0.0', 4, 0, 'http://mach-ii.com/MachIIDashboard_1-0-0_Beta.zip', '2009-02-09', '2009-02-09', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(11, 'dashboard', '1.0.0', 3, 1, 'http://mach-ii.com/MachIIDashboard_1-0-0_RC1.zip', '2009-04-15', '2009-04-15', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(12, 'framework', '1.8.0', 5, 1, 'http://mach-ii.com/MachII_1-8-0_nightly.zip', '2009-04-22', '2009-04-22', NULL);
INSERT INTO "activePackages"("packageID", "packageName", "version", "releaseType", "packageStatus", "fileLocation", "dateReleased", "dateUpdated", "build")
  VALUES(13, 'dashboard', '1.0.0', 5, 1, 'http://mach-ii.com/MachIIDashboard_1-0-0_nightly.zip', '2009-04-16', '2009-04-16', NULL);

I thought it might be a good idea to keep all the data in the table and indicate which is the "active" version for each release Type. Here are the release types I have:

1 = Stable
2 = RC2
3 = RC3
4 = Beta
5 = Nightly

Once I have my steps on how the DB should be updated written I will post for your review.

[derrickjackson]

I'd like to talk more about this when you have time.

[derrickjackson]

Here is how I see the DB being updated. I will use M2 1.8 as the example:

1.  Nightlies:  When a new nightly is published the database is updated with the following information:  fileLocation, dateReleased where packageName = framework and version = 1.8.0

2.  When 1.8 moves to Beta.  1 of 2 things can happen to the nightly entry:  A.  Nightly status changed to 0 to indicate there are no new nightlies OR B.  Nightly entry for the last updated 1.8 nightly stays the same.  Whatever the decision for the nightly for 1.8 is, the following happens for creating the beta record for 1.8.  The record is inserted into the DB (packageName, version, releaseType, packageStatus, fileLocation, dateReleased, build) VALUES ("framework", "1.8.0", 4, 1, "http://mach-ii.com/code/fileName", "DATE", xxxx).  After this is done, an UPDATE is made to find all Beta versions of the same package type and set their status to 0.  Can't have more than one active beta for the same package.

3.  When 1.8 moves to RC1.  Insert the new record, then make sure the beta for this same version is no longer active.  Then set all other RC1s in the same package to 0.

4.  When 1.8 moves to RC2.  Repeat steps from 3, but replace the release type.

5.  When 1.8 moves to Stable.  Insert new record for this stable release.  Update all other stable release types for this package type and set their status to 0.  Do the same for and RC2s for this package type.

6.  Finally when any of these is done, cache the data for an hour.

7.  User's dashboard hits the following URL (or one similar):  http://mach-ii.com/?event/[eventName]/package/[packageName]/version/[versionNumber]

Hopefully this makes sense. I will try to drop in a screen shot of what I have done to dashboard.

[derrickjackson]

here is the sample phone home image (hopefully)

[peterfarrell]

Only have a moment here Derrick, please excuse my brevity. FYI, nightlies are usually publish during the Beta, RC1, etc. phases of the release schedule as well. There are "nightlies" and the more "official" releases (beta, RC1, RC2, stable, etc.). The nightlies are discontinued when an official stable is released.

The fourth digit of the release indicates a release number. See FAQMachIIVersion for more information. For example 1.8.0.8 indicates the "stable" release, while 1.8.0.1467 would indicate a nightly release of 1467 (the last commit number for that nightly). A release of 1.8.0.0 means it was pulled from SVN and therefore no way for us to check the release version (this means you'll probably not be able to auto-update when the release is at this level).

[derrickjackson]

gotcha on the nightlies part. Once i start writing the methods to take care of all the steps above I will take into account that nightlies are published until we get to the stable release type.

Not sure where "FAQMachIIVersion" is located. Will search the WIKI for it so I can better understand this.

Thanks for the quick info and will chat more when you have time. Thank again.

[peterfarrell]

Derrick, do you have access to the website repo? It's not public, but I believe I let you have access to it:

 http://greatbiztoolsllc.svn.cvsdude.com/mach-ii/website/

Let me know and you can start building the "server" side event-handlers for mach-ii.com. I'll add you if you don't have access.

Sorry, missed it as a link - FAQMachIIVersion.

[derrickjackson]

I just checked and I was denied access to the repo. Also I found the FAQ WIKI page and now I am on the same page as you with releases. I guess i never noticed the fourth number in a release. This is all starting to come together.

[peterfarrell]

Derrick, just updated your perms to give you access to website module. It might take a while for it to propagate.

[derrickjackson]

I now have access... Thanks Peter.

[derrickjackson]

Peter do you think the database I recommended above will work? I plan to start building the mach-ii.com portion today and having the DB in place would be a great help. If not then I will build all the other parts around it.

[peterfarrell]

Yeah, I didn't look too closely at the DB part, but I think it will work. I think we can experiment a bit here and refine stuff latter. So go ahead and work on it. BTW, mach-ii.com is using MySQL as the DB engine. Also, you might change date updated and date change from DATE to DATETIME so we can know the time of day (we something do more than one nightly a day - rarely - in case big bug is found in the nightly).

[derrickjackson]

Ok... I downloaded the M2 website files from SVN just now. Is there anything special I need to know about setting up the website so I run it locally? I noticed there isn't an index.cfm or application.cfc file in the main directory.

Also, am I able to create the DB or are you saying you will create it?

[peterfarrell]

We use a more secure method of setting up our applications. The index.cfm and Application.cfc are in the /public directory. Have your virtual host point to that directory. Set up a CF mapping called m2website to point to the root of the application (not the public directory but the directory that has the model, views, config folders sitting in it). That should be everything to get it running... and Mach-II 1.8 BER as well.

As for the DB, we don't have one yet (except for the blog) so just go ahead and create a DB creation script for that.

[derrickjackson]

Peter,

I emailed the table creation script to your M2 email address.

[derrickjackson]

Ok... My changes/additions are "complete". I will go over all of them in two sections: Dashboard 1.1.0 and M2 Website additions.
Dashboard 1.1.0
Added the following items to the config file:

<!--Release Types (Will be used in the Auto Update dashboard tab)-->
<property name="autoUpdateReleaseTypes">
<struct>
	<!--The first element when pulled from SVN is 0 indicating you are using a BER with an unknown build number.-->
	<key name="P0"  value="BER"/>
	<key name="P1"  value="Alpha"/>
	<key name="P2"  value="Beta"/>
	<key name="P3"  value="RC1"/>
	<key name="P4"  value="RC2"/>
	<key name="P5"  value="RC3"/>
	<key name="P6"  value="RC4"/>
	<key name="P7"  value="RC5"/>
	<key name="P8"  value="Development - Stable"/>
	<key name="P9"  value="Production - Stable"/>
</struct>
</property>

<!---This listener contains one method (getPackageData) which uses cfhttp to call out to the M2 Website and get the available packages.  
The user does not have to pass in their version of M2 or Dashboard.  I am using available functions to get this data--->
<listener name="autoUpdate" type="MachIIDashboard.listeners.AutoUpdateListener"/>

<!-- auto update -->
<event-handler event="autoUpdate.index" access="public">
	<event-arg name="meta.title" value="Auto Update Information"/>
	<event-arg name="meta.refresh" value="5" />
	<notify listener="autoUpdate" method="getPackageData"/>
	<view-page name="autoUpdate.index" contentArg="layout.content"/>
	<execute subroutine="doLayout"/>
</event-handler>

<!-- info -->
<page-view name="autoUpdate.index" page="/views/autoUpdate/index.cfm"/>

NOTE: In the service layer (AutoUpdateService?.cfc) I am hard coding the server name to be test env (m2website). Maybe we can use ANT to replace that value when the website build is done.

Mach-II Website
There is a new directory that will need to be added to the M2 webserver mappings: autoUpdate. This folder contains the files necessary to provide the package data for the dashboard as well as the administrative function of updating the package data database.
Added the following to the config file:

<includes>
	<include file="/autoUpdate/config/mach-ii_autoUpdate.xml.cfm" /> 
</includes>

<!--Added this so the autoUpdate would have its own caching structure for the 1 hour cache of package Data-->
<parameter name="autoUpdate">
	<struct>
		<key name="type" value="MachII.caching.strategies.TimeSpanCache" />
		<key name="cachingEnabled">
			<struct>
   		    	<key name="group:development" value="false"/>
   		    	<key name="group:production" value="true"/>
			</struct>
		</key>
		<key name="scope" value="application" />
		<key name="timespan" value="0,1,0,0" />
		<key name="cleanupIntervalInMinutes" value="3" />
	</struct>
</parameter>

Added the following to the Coldpsing config file:

<!-- used for the auto update feature-->
<bean id="datasource" class="autoUpdate.model.datasource.Datasource">
	<property name="dsn"><value>autoUpdate</value></property>
</bean>

<bean id="autoUpdateService" class="autoUpdate.model.autoUpdate.autoUpdateService">
	<property name="autoUpdateGateway"><ref bean="autoUpdateGateway" /></property>
	<property name="autoUpdateDAO"><ref bean="autoUpdateDAO" /></property>
</bean>
	
<bean id="autoUpdateGateway" class="autoUpdate.model.autoUpdate.autoUpdateGateway">
	<property name="datasource"><ref bean="datasource" /></property>	
</bean>
	
<bean id="autoUpdateDAO" class="autoUpdate.model.autoUpdate.autoUpdateDAO">
	<property name="datasource"><ref bean="datasource" /></property>	
</bean>

There is a new folder in the views section called "autoUpdate". It contains the page that processes/receives the wddx data from the cfhttp called made by dashboard. I tried to add this file to the autoUpdate folder that was used in the <include> above, but could not figure out the mapping in the mach-ii_autoUpdate.xml.cfm config file.

This line of code <page-view name="versionData" page="/views/autoUpdate/index.cfm"/> from the maci-ii_autoUpdate.xml.cfm config file actually points to the autoupdate folder within the M2 views section.
NOTE: The autoupdate feature cannot run on its own. It is part of Dashboard.

Now that you know the meat of what I changed/added, here is how I see everything working.
User/Client? Side:

1. User adds the dashboard to their application. Based on their version of M2 and Dashboard.
   
2. When they click on the "Auto Update" tab, a cfhttp request is sent to the Mach-II website.
   
3. This request returns package data for the framework and dashboard. The package data will contain information for their current release of the package as well as the nightly for each.
   
   

Administrative Side:

1. There one public event (getPackageData) and one "private" event (updatePackage) that is "protected" by a filter and requires a password to be called.
   
2. If a user/client tries to call the event without the password or the wrong password he/she will be redirected to the mach-ii website with no explaination.
   
3. When a new build is ready for the framework or dashboard an ANT task will call the updatePackage event and pass in the following arguments: packageName, version, releeaseType, fileLocation, and password).
   
4. Before and update/insert is done to the database, a check is done to see if the pacakge already exists. If it does it updates the package. If not is inserts the package. Nightlies(framework/dashboard) are the only packages that are updated. Based on what Peter mentioned earlier. Once a package goes to any of the other release types (alpha, beta, RC1, etc) changes to them are done in nightlies.
   
5. When Mach-II 1.8 is ready to go to Alpha release the following will happen:
   
   1. A new record will be instered into the database and this new record will be given a status of 1 to indicate it is active.
      
   2. An update will be made to all other versions of 1.8. Since at this point the only other version is a nightly its status will be changed to 0 to indicate at this point there are no active nightlies.
      
   3. Now if after going to alpha a nightly is published the status for this nightly change to 1.
      
   4. This process will continue up until the release goes to stable.
      
6. Once the update/insertion is made the package cache is cleared.
   

We should probably talk about this next week at cfObjective just to be certain we are on the same page. However thoughts before then are welcome.

[derrickjackson]

One more thing. I have not committed any of the M2 Website changes yet. I wanted to be sure you were ok with my changed before I did so.

[peterfarrell]

All looks good to me Derrick!

[derrickjackson]

great... i just posted my changes to the M2 website SVN location.

[peterfarrell]

Derrick, does the autoUpdater package need an Application.cfc and index.cfm? It doesn't appear to be that way since it is an include.

[derrickjackson]

it sure doesn't. Thanks for catching that.

[derrickjackson]

I see you made some code "comments". LOL...

[derrickjackson]

Hey Peter,

I see Dashboard 1.1 is now BER. Did you ever get a chance to build the portion that updates the database when a new build is published?

[peterfarrell]

Dashboard development is going to be merged into main framework trunk

[derrick_jackson]

pjf,

is there a replacement URL for this link: http://www.mach-ii.com/index.cfm/go/getVersions/

[matt_woodward]

 https://github.com/Mach-II/Mach-II-Framework/issues/34