Databases: 2.0 API params annoyance (I think)

Databases: 2.0 API params annoyance (I think)

by Howard Miller -
Number of replies: 9
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
Why is this an error....

$test = $DB->get_records_select(
'user',
'firstname=:name or lastname=:name',
array('name'=>'fred'));

but this isn't

$test = $DB->get_records_select(
'user',
'firstname=:name1 or lastname=:name2',
array('name1'=>'fred','name2'=>'fred'));

??

Looking at the code, it seems to insist that there are the same number of parameters supplied as there are parameters in the sql - regardless of any of them being the same parameter (name) or not. Bug/feature/intentional?


Average of ratings: -
In reply to Howard Miller

Re: Databases: 2.0 API params annoyance (I think)

by Petr Skoda -
Picture of Core developers Picture of Documentation writers Picture of Peer reviewers Picture of Plugin developers
No, this is intentional because else you might override one parameter with another and not know it in long queries, sorry this is much safer.

Also sql standards do not allow this, we would have to rename the parameters somehow.

Petr


In reply to Petr Skoda

Re: Databases: 2.0 API params annoyance (I think)

by Eloy Lafuente (stronk7) -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers
Hi,

well, I agree about being possible to override them (by error) in long queries and so on, but also think it's possible (easier) to lose the order of "?" parameters (by error too), and that isn't preventing us to use them. wide eyes

In any case (overridden named params or wrong order of ? params), it's a developer bug, like any other bug, and should be fixed. I don't see the "safer" point here really.

Also, talking about SQL standards... uhm... perhaps we should exclusively allow "?" parameters if we want to adhere 100% to it? If I don't remember wrong, from my old SQL sessions... question marks were the official ones.

I think I'll be using "?" all the time... but really the "repeated named ones" sound logic and natural... unless some database driver forbids it. Although that can be bypassed by renaming params internally if necessary.

uhm... uhm... ciao smile
In reply to Eloy Lafuente (stronk7)

Re: Databases: 2.0 API params annoyance (I think)

by Petr Skoda -
Picture of Core developers Picture of Documentation writers Picture of Peer reviewers Picture of Plugin developers
some dbs require ? some require :names yet others require $1 wink

http://bugs.php.net/bug.php?id=33886 explains the repeated :named params
In reply to Petr Skoda

Re: Databases: 2.0 API params annoyance (I think)

by Eloy Lafuente (stronk7) -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers
> some dbs require ? some require :names yet others require $1 wink

yes, yes. that's the reason we are processing queries and converting the original placeholders to the required ones for each DB. The result: we support both type of placeholders against all DBs (absolutely amazing!).

In the same direction... query processing could handle those repeated names too, splitting them and adding param internally IMO. One cool plus-plus for the API.

Ciao smile
In reply to Eloy Lafuente (stronk7)

Re: Databases: 2.0 API params annoyance (I think)

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers
To take a slight tangent here.... I'm concerned that this is going down the route of producing very unreadable (SQL) code. Just my personal opinion, but I think clear, readable code is right up there as one of the most important things you can strive for.

First thing is that I really don't understand why you would want to use (or for that matter, allow) the use of a ? as a param placeholder - it infers no meaning whatever in the SQL statement. Secondly, not allowing repeated parameters where repeated parameters are *logically* what the SQL writer intended again obscures the sense of the SQL statement. It's just *asking* for tricky bugs to be introduced.
In reply to Howard Miller

Re: Databases: 2.0 API params annoyance (I think)

by Eloy Lafuente (stronk7) -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers
Well,

both types of parameters are supported. That's cool. 100%

I commented in my previous post that I (personally) was going to use the ? parameters mainly because of:

- ? is the SQL standard (AFAIK)
- I'm very used to use those ? (java developer here).

Anyway, they great thing is that if I need that, I can change and use named parameters instead (when one query becomes really big, or when it's constructed dinamically, based in different conditions).

Personally, I don't think ? params aren't readable by definition. Not at all. Instead I'm pretty sure that the "real" usage of the new DB layer will give us the best (more readable, less prone to errors...) way to build each query.

About your second part, yes, I agree that allowing repeated parameters sound logical (and natural) in my mind. No discussion here.

Ciao smile
In reply to Petr Skoda

Re: Databases: 2.0 API params annoyance (I think)

by sam marshall -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers
I'm not convinced it is 'safer' but it makes queries harder to build when you are constructing a complex query in code. For example, here is what I just had to do to a query in lesson:

 $params = array("lessonid" => $lesson->id,"lessonid2" => $lesson->id);

 if (isset($userid)) {
 $params["userid"] = $userid;
 $params["userid2"] = $userid; $user = "AND u.id = :userid";
$fuser = "AND uu.id = :userid2";
 }
 else {
 $user="";
 $fuser="";
 }

 if ($lesson->retake) {
 if ($lesson->usemaxgrade) {
 $sql = "SELECT u.id, u.id AS userid, MAX(g.grade) AS rawgrade
 FROM {user} u, {lesson_grades} g
 WHERE u.id = g.userid AND g.lessonid = :lessonid
 $user
 GROUP BY u.id";
 } else {
 $sql = "SELECT u.id, u.id AS userid, AVG(g.grade) AS rawgrade
 FROM {user} u, {lesson_grades} g
 WHERE u.id = g.userid AND g.lessonid = :lessonid
 $user
 GROUP BY u.id";
 }
 unset($params['lessonid2']);
unset($params['userid2']);
} else { // use only first attempts (with lowest id in lesson_grades table) $firstonly = "SELECT uu.id AS userid, MIN(gg.id) AS firstcompleted FROM {user} uu, {lesson_grades} gg WHERE uu.id = gg.userid AND gg.lessonid = :lessonid2 $fuser GROUP BY uu.id"; $sql = "SELECT u.id, u.id AS userid, g.grade AS rawgrade FROM {user} u, {lesson_grades} g, ($firstonly) f WHERE u.id = g.userid AND g.lessonid = :lessonid AND g.id = f.firstcompleted AND g.userid=f.userid $user"; }
That's a lot of opportunities to screw up (and maybe I did - if you spot one don't tell me, just fix it, please smile MDL-15877). The code is a whole lot nicer if you're allowed to reuse them, imo. Chance of wanting to do that is hugely greater than the chance of you constructing a long query with two different :userid parameters that you don't notice are the same, imo.

The database support issue is not relevant: Moodle already does loads of stuff before this gets anywhere near a database, right? It would be trivial to create a preprocessing function

munge_multi_params(&$sql,&$params)

that automatically expands the variable names to add a 2 or whatever in the sql and the $params.

All this said, I think it's ok to leave as is if we are really convinced it helps safety, but there needs to be an error message that hits this specific case because everybody is going to do it wrong, even if it's in the documentation. At present you get invalidparams. Instead it should be something like:

'Your database query <pre>SELECT * FROM whatever WHERE x=:apple AND y=:apple</pre> uses the same named parameter <b>:apple</b> more than once. This is not permitted. Please correct your query.'

(Could/probably should be developer debug instead of an actual print_error; print_error could use a generic as at present.)

--sam
In reply to sam marshall

Re: Databases: 2.0 API params annoyance (I think)

by Petr Skoda -
Picture of Core developers Picture of Documentation writers Picture of Peer reviewers Picture of Plugin developers
Hmm, using the same param twice in one query?

SELECT * FROM {table1} t1, {table2} t2 WHERE t1.user = :userid AND t2.userid = :userid

Ok, I know this is silly example, but I think it is often possible to rewrite the code with one :userid only, right? The only use case I found so far are complex OR conditions, did I miss anything else?


The problem with repeated named params is that you might get query parts from different places, in that case you would be warned that something is wrong. I was converting most of the code and saw around 5 places that needed repeated params. I believe that the benefit of safety is much bigger compared to 5 more params in our codebase.

Petr