DataMapper 1.6.0

[eluser]stensi[/eluser]
@OverZealous.com: Well, maybe partly, lol

Mostly though, I changed $fields because upon further investigation, I found the returned meta data was quite different depending on the type of database being used. I originally envisioned using it for help with automatically determining a fields data type for display in HTML, but even if the data was returned in a standard way for all databases, I don't think it would have worked out anyway. For example, a value might be stored as an integer but the developer might want it displayed as currency, or you could have a unix timestamp stored as an integer, and they wanted it displayed as a date. Better to have the developer specify the data type in the validation array.

Besides that, I find the code just looks cleaner when foreach'ing through just the fields, lol

Ah, the joys of server management. I recently got a slice at SliceHost.com and am setting it up at the moment.

@empoleon: Could you post the $has_many and $has_one settings you have in each of your Job, Company and Product models?

[eluser]empoleon[/eluser]
Here are the relevant tables:
// -----------------------------------------
// NORMAL TABLES
// -----------------------------------------
companies (3 records)
id
company_name

jobs (3 records)
id
job_date_created
job_date_completed

products (6 records)
id
product_name
product_model
product_sku

// -----------------------------------------
// JOINING TABLES
// -----------------------------------------
companies_jobs (3 records)
id
company_id
job_id

companies_products (6 records)
id
company_id
product_id

jobs_products (6 records)
id
job_id
product_id

// -----------------------------------------
// MODELS
// -----------------------------------------
class Company
var $table = 'companies';
var $has_many = array('job', 'product');

class Job
var $table = 'jobs';
var $has_many = array('product');
var $has_one = array('company');

class Product
var $table = 'products';
var $has_many = array('job'); // A 'Product' can be on different 'Jobs'
var $has_one = array('company');

[eluser]OverZealous[/eluser]
@stensi - 1.5.3 'upgraded' easily enough ;-)

Thanks for mentioning slicehost - I had heard about them a while back, but had forgotten. Amazing prices! I had gotten a quote from Rackspace - since I need a certain level of control over the server - and they start at almost $400/month. With the upgrade-ability of slicehost, I could probably start on a much lower plan, and upgrade as I get more paying customers!

I currently, and for the short term future, am hosting the application out of my home on a business connection. (Of course, network uptime is sometimes an issue, even if bandwidth, storage, and server management are not.)

[eluser]stensi[/eluser]
@empoleon: The number of queries you get will depend on how many records there are in each table. I'll try to explain which code causes which queries:


This one is run to determine if a corresponding table exists for each of your models:

Code:

SHOW TABLES FROM `your_database_name`

These are run once per model to get the list of field names:

Code:

SELECT * FROM `jobs` LIMIT 1
SELECT * FROM `companies` LIMIT 1
SELECT * FROM `products` LIMIT 1

This one...

Code:

SELECT * FROM (`jobs`)

...is executed when you do:

Code:

$j->get();

This one...

Code:

SELECT `companies`.*
FROM (`companies`)
LEFT JOIN `companies_jobs` ON `companies`.`id` = `companies_jobs`.`company_id`
LEFT JOIN `jobs` ON `jobs`.`id` = `companies_jobs`.`job_id`
WHERE `jobs`.`id` = 1

...is executed when you do:

Code:

$job->company->get();

You'll get a similar query each time you loop through the 1st foreach, for each job ID, so you might have multiple ones like this.


Note that I don't think your show_jobs() method ever gets into the 2nd foreach because you're missing a get() on the $job->product. It should have:

Code:

foreach ($job->product->get()->all as $product)

This causes this type of query:

Code:

SELECT `products`.*
FROM (`products`)
LEFT JOIN `jobs_products` ON `products`.`id` = `jobs_products`.`product_id`
LEFT JOIN `jobs` ON `jobs`.`id` = `jobs_products`.`job_id`
WHERE `jobs`.`id` = 1

Again, you'll get a similar query each time you loop through the 1st foreach, for each job ID, so you might have multiple ones like this.


This one...

Code:

SELECT `companies`.*
FROM (`companies`)
LEFT JOIN `companies_products` ON `companies`.`id` = `companies_products`.`company_id`
LEFT JOIN `products` ON `products`.`id` = `companies_products`.`product_id`
WHERE `products`.`id` = 1

...is executed from this code inside your 2nd foreach:

Code:

$product->company->get();

In one of the coming versions, I plan on including "related_{clause}" methods to help developers fine tune their queries on related objects, to reduce the total number of queries needed. At the moment, doing the type of thing you have in your show_jobs() method might indeed generate a fair amount of queries, if there are lots of records. Even so, databases are designed to handle JOIN's very quickly so will cope fine with this sort of database querying.

I'd recommend you page your results though, rather than just showing everything, which will save on unnecessary bandwidth usage. Example:

Code:

$page = 4;

$limit = 10;
$offset = ($limit * $page) - $limit;

$j = new Job();
$j->get($limit, $offset);

That will return a maximum of 10 records, starting at page 4 (so records 40 to 50). You can easily do this with your related items as well.

In most cases, you wouldn't show all of the related information of your jobs when just showing the list of jobs as you're wasting bandwidth doing that. You should only show data when the user specifically needs or asks for it. For example, IMHO there's no need for that 2nd foreach you have. Instead, I'd list the jobs with the company name beside it and only show the list of products relating to a job after they've clicked the job for the details. Otherwise, you have all this product information loaded in other jobs, when they might only care about the products of that one job. You know what I mean?


@OverZealous.com: Yeah, slicehost is nice and cheap for the amazing service they provide. I'm starting off with a 256MB slice until I get everything setup and see what the performance logs say. It's good to be able to easily grow your slice as needed and their biggest one sure is meaty, lol!

Although it's very hands on and can take up a fair amount of time to configure and maintain, the level of control you get far out ways the downsides for me. It's still early days for me on there though so I'll let you know how I find it in the longer run.

[eluser]empoleon[/eluser]
@stensi: I corrected my original post-- you pointed out my method was missing a query I'd neglected to paste in:

$job->product->get();

You're right on about presenting only the data the user has requested or needs at the time (i.e. removing the 2nd 'foreach' loop). I was thinking a bit about those special people in the workplace who need ridiculously verbose reports and how Datamapper would manage such queries. You reminded me that our server can manage the joins-- I'll crack on then.

Thanks for your excellent explanation. It really cleared things up for me.

[eluser]OverZealous[/eluser]
@stensi

Bug in new save method:

The new save method has a fairly serious bug. Because it tries to save the object every time save is called - even when an object is passed in, new objects that are saved to the database get re-saved multiple times.

Example:

Code:

$user = new User();
$user->name = 'Bob';
$user->email = '[email protected]';
$user->save();

$group = new Group();
$group->get_by_name('user');

$user->save($group); // $user gets REsaved to the database here.

Why does this happen, you ask? It's because the $stored array is populated with the original (all NULL) data. However, the new object now has an ID, a name, and an email (in the example). Therefore, it always appears to have changed. Obviously, the $stored array needs to be updated after a successful save.

The real issue, however, came when I tried to save something with NOT NULL datecreated and dateupdated fields. For some reason, DM is trying to save them as NULL. I'm still looking into this one.

UPDATE: Found that sucker! Your tests to see if the create_date and updated_date fields is still stuck on array_key_exists, from DataMapper 1.5.2!

As for the other, here's my fix, but it's too frickin early, and I need some sleep, so I didn't get a chance to test it thoroughly. I think it's still saving them multiple times.

Code:

// immediately after insert or update:
// populate $stored
foreach($data as $field => $value)
{
    $this->stored->{$field} = $value;
}

// only in insert:
// Assign new ID
$this->id = $this->stored->id = $this->db->insert_id();

Also, I think you made a mistake in the test to see if the only changed field was $updated_date. It's inside the unset loop, and I think it's supposed to be after that loop.

[eluser]stensi[/eluser]
Ah, thanks very much for that! You're right on all accounts.

Just to note though, when saving you can do both the INSERT/UPDATE of a new object at the same time you save a relationship onto it. So, this line is not necessary:

Code:

$user->save();

Due to the fact you have this line:

Code:

$user->save($group);

With a properly working version of DM, there's no issue with having both lines of save code there though, since the first would insert the user and the second would see the user itself has not changed and so only insert the group relationship (if any user fields had changed though, they would update at that time).

You could leave both your save calls in there and add the following check before saving the relationship:

Code:

if ($group->exists())
{
    $user->save($group);
}

That'll save on a bit of extra validation processing.

I've fixed up all the issues you mentioned except for the NOT NULL/NULL thing which I'll try to replicate now.

[eluser]OverZealous[/eluser]
The NULL bug should be fixed by changing the created/updated_date line. It was just skipping them, so it saw them as NULL values.

The reason I can't do it all at once, in my code, is I do some pre-processing for security reasons. I need to be able to save the data, verify it, save the next item, verify it, etc. Actually, I'd almost rather have the option to disable the auto-save.

Would it make sense to add a flag on the save method to skip saving this object?

Code:

function save($object = NULL, $save_this = TRUE)
{
    if($save_this)
    {
        // save this object
    }
    if( ! is_null($object))
    {
        // save relationship
    }
}

Also, I'm dynamically processing some JSON-encoded fields. Once you leave the realm of normal HTML forms, things get weird. ;-)

[eluser]stensi[/eluser]
Ah, I was wondering why I wasn't able to reproduce it, lol

I benchmarked the save to see the impact and, if there are no changes to the object between a save without a relationship and a save with a relationship, then there is no noticeable impact. The comparison between the field and the stored field happens very fast and skips over the more involved validation processes, so yeah, it's fine the way it is :-)

I've fixed up everything now and have improved the use of the CI Form Validation library to instead load it normally rather than creating instances of it.


_________________________________

Version 1.5.4 has been released!

View the Change Log to see what's changed.

In short, the stored values are now being refreshed correctly during save()'s and a silly bug was fixed (thanks to Phil for bringing these to my attention). There's also a minor improvement to the use of the CI Form Validation library.

Enjoy :-)

[eluser]aztack[/eluser]
Great Job stensi~~ That's is what I need! Thanks!