Forward search queries to websites or other Apps.

Hi all,

I would like to propose a new feature for the search in fdroidclient.

The feature I have added allows the user to forward search queries to external sites or apps, for example to Google Play or the web browser.

The intention behind the feature was that getting "No applications were found matching ..." as the only search result is somewhat disappointing. Especially, when you know that the app your are looking for exists (within F-Droid?). Since the search function in the client is not really intelligent (no offence ) it also does not return any results when one misspells an app's name or is searching by a keyword that is not present in the database. While enabling the search to handle typos in the query should be possible to some extend, making the search "intelligent" and return correct results for keywords that are not part of title or description is somewhere close to impossible. At least without a much bigger knowledge base...

Therefore, integrating an option that lets the user select an external search would improve the user experience (no more "No app"-one-ways) while keeping changes to the code relatively small.

Destination of a forwarded search can be any URI in which the search query is part of the URI (e.g. the query string).

Note: The code is already working (as in: it does what I expect it to do and does not break anything else) but there are still unnecessary debug outputs, etc included and the code itself could be cleaned up a bit. Maybe some wired error cases are not handled (I did not find any while clicking around...)

But since I don't really know when I will find time to finish it, I decided to open a merge request as long as it is in sync and can be merged in cleanly.

---

Changes to the user interface

• New menu item "Manage external search sites"
• Management Activity for the search sites, it allows defining new sites, editing existing ones and deleting them
• Added tabs to the SearchResult Activity. The first tab is the "normal" list of search results, the second one the list of external sites.

---

Changes in the code

I tried to stay as close as possible to the style and structure of the existing code and to reuse as much as possible. Some parts of the existing code I copied into the new classes to allow independent changes. Some functions I factored out into the Utils class.

I reused the TabManager for the tabs in the search results. This change also required a new TabSelectionListener interface, that is now used by the FDroid main app and the SearchResults.

External search sites are managed in a new database table "fdroid_external_search". An initial set of search sites (Google Search, Play Store, Amazon Store, DuckDuckGo) is added initially to the database.

The Database version in the code is set to 52, the code assumes 52 to be the version that introduced this feature.

Edit: reduced image size and moved them out of the text

This doesn't seem like a good idea to me for the following reasons:

• It clutters the UI.
• It makes F-Droid no longer reliable as a place you can get Free apps from, as the external search results will list non-Free apps.
• It implicitly recommends people to download and install random APKs from the Internet, which is a very bad security practice.
• It recommends people to, in case F-Droid doesn't have an app (most likely because it is non-Free), to install it from Google Play, which actively undermines F-Droid's purpose.
• It is out of scope; F-Droid is an application manager, not a search engine manager. Apps like Fennec already have this functionality and are a much more logical choice when you want to search for things outside of F-Droid.

I haven't looked at the code, but these are my two cents. I apologize for sounding harsh, your intentions are most likely positive, but I just do not consider this a good idea at all.

I'm also unclear on how this helps find apps on F-Droid. For all I can see, this is like a web browser search toolbar, which doesn't really fit into F-Droid.

For issues on how to improve our current search engine, see #336.

I have been expecting most of your arguments to come up. They crossed my mind when I started to think about writing it, too. Therefore, I understand your concerns. However, I decided to write it after putting some thought in if and how it could integrate with F-Droid or if it would mess up the app's ideals/appearance/reputation/... because I actually do see a benefit for the F-Droid app. Let me try to explain.

Even though most of the points @TheLastProject mentioned boil down to "F-Droid is a place for free apps", I will try to differentiate:

• It clutters the UI.

Yes, the UI integration is far from perfect. In fact, the menu item for the manager activity rather fits in the settings than in the main menu. Do you also think the search results view is "cluttered"? If so, why do you think so? There is a clear separation of everything else, the tabs are clean F-Droid style and it does not mix things...

The description field can be removed it turned out to be quite useless anyway. This would clean up the appearance a bit.

• It makes F-Droid no longer reliable as a place you can get Free apps from, as the external search results will list non-Free apps.

First of all, it does not change the fact, that F-Droid is a place for free apps. This was the reason for me not to integrate the results of an actual search directly into the UI. Integrating results of a search, e.g. performed on google play, would really give the impression that F-Droid now "markets" non-free apps. Plus the bad user experience you would get when you have would to explain, why you can search but not install apps from Play... Second, both, the external search tab which is somehow hidden until you explicitly click it and the prompt to select an app in the "complete action" prompt are a clear borderline where the responsibility of F-Droid ends.

Anyway, yes, it could be wise not to include Play or the Amazon store as default. I just wanted to show what could be done with it and a user who needs it can still add it himself.

• It implicitly recommends people to download and install random APKs from the Internet, which is a very bad security practice.

True. Period.

But I don't see how this would be an argument against this feature. Yes, it will do the copy&paste work of the search query into the browser for the people but it will not help them to install that random APK.

• It recommends people to, in case F-Droid doesn't have an app (most likely because it is non-Free), to install it from Google Play, which actively undermines F-Droid's purpose.

Well, we could start a discussion about where "recommending" begins... I see it as a simplification of interacting with the app (see use-cases below). The way I see F-Droid client's purpose is that it searches, manages and updates free apps. The purpose I see in F-Droid as such is to offer a platform to free apps and developers and users who don't want to support Google's money making machine. I assume, we are on the same level on this one? I don't think that you could keep users from installing apps from Google Play or wherever if they think they need to. And you will motivate even less users to write their own free alternative. Unfortunately.

• It is out of scope; F-Droid is an application manager, not a search engine manager. Apps like Fennec already have this functionality and are a much more logical choice when you want to search for things outside of F-Droid.

This is close to what @mvdan said comparing the feature to a browser search toolbar, which is somehow the case. Haven't seen it that way so far...

The option to "manage" search sites was intended to allow the user to configure which engine or app he wants to use. Maybe the moving the main menu entry to a less prominent position in the settings activity could help.

Yet, the "logic" depends on where and with what intention you start opening an app. You open the browser to search something in the web, read news, whatever. Usually you don't search apps in the browser when you have an application manager. On the other hand, you open F-Droid to find and manage apps. If you don't find what you are looking for, you switch to the browser and search again. This is where this feature comes into play: It links F-Droid to a web-search (or $store) and simplifies what the user would probably do anyway. Think of it as a shortcut and/or as the opposite of F-Droid's ability to open links pointing to another app store (e.g. when you click a link going to Google Play).

Writing this leads me to the assumption that I did not explain my intention and the connected use-cases well enough.

First, and just to stress it again, the external search tab only shows a list of possible destinations where to send a query to. It does neither act as a web browser by itself nor list apps that are not present in one of the client's repositories nor help to install apps. It simply opens an intent with the query encoded and inserted into a pre-configured link.

Second, I am 100% positive that F-Droid is not a place to advertise non-free apps. Any mix of free and non-free apps within the client would weaken what F-Droid stands for. This was the reason for me, for example, not to open the "external" tab directly when the search returns no results. You will still have to click it.

Third, the searches I have included are exemplary. It does not need to be a place where you could actually download apps, it can be anything from search engines to app review sites or whatever...

To make it a little more clear how F-Droid could benefit from this feature, here two use-cases, I'm sure there are more.

Use-case 1:

(My use case...) On my pretty old and slow phone I use F-Droid as the only App provider. It's all cool but switching apps via apps drawer or the recently opened apps is painfully slow. Therefore, I am happy if an app switch is done by an intent and directly reaches its destination.

When I search an app that I have heard of and I don't remember the name I try it with a keyword or to guess the name. Many times it does not turn up any results. Without the feature I then switch to a browser, enter the same search query again and wait for the search engine to have it spell-checked for me and dig up some results. Usually one of the first results is the app I'm looking for in Google Play. I click the link, and F-Droid opens it.

With the feature, I select one of the search engines, let it be Google, which opens in the browser and directly returns the results I was looking for. I click the Google Play link, done.

More concrete example: I have been looking for OpenTasks (formerly Tasks). I knew it is in F-Droid but searching "opentasks" does not return any results. I missed the fact that it was simply called Tasks before and opened a browser to search it. Play is the second result in DuckDuckGo, probably the first in Google. I followed the Play link, F-Droid resolved it for me and opened the app with the title "Tasks" and no "OpenTasks" in the text. The feature would have spared me from a lot of unnecessary clicking around.

Use-case 2:

Many users of F-Droid I know use it as their main source of apps but have Google Play installed beside of it. They have (or at least think they have) their reasons to do so but that's not the point here. The point is, when they search an app that is definitely not in F-Droid, maybe because its non-free, they switch to Google Play, search it again and install it from there. From my experience, (most) users tend to be lazy and prefer what is convenient for them. Searching the same thing twice and getting a disappointing "No applications found" in between is not convenient. Maybe they will try again once or twice but slowly turn back to Play where they "can get most apps F-Droid has" plus the non-free apps they (think they) need. I actually think, this feature could help to maintain F-Droid's position as the primary source of apps on their phone since they will not actively have to switch apps and search again.

---

I hope you can see some benefit for the F-Droid client now. Maybe not for yourself but for other users. I know it sounds a bit as if I wanted to convince you to integrate this feature. But it is useful to me so I wanted to share it because I think it has use for others, too. Additionally, it only does what most users will do anyway when they get no search results. So why not helping them?

Ups, didn't feel that long when I wrote it, sorry for this wall of text Hope you'll read it...

Another thing that I just thought of: maybe renaming "external search" to "web-search" would make the purpose of it more clear.

Sorry for leaning in, I am aware that I have no credibility in this community at all. While the intentions are good, I consider this the killer argument: "it facilitates people finding and downloading random apps from the internet".

For me, fdroid is all about providing trust in the fact that I can inspect corresponding source codes. If apps pop up when searching fdroid that are completely untrustworthy, this advantage will be lost. And others, will for sure misunderstand and mix things up: " but I got this malware app from fdroid".

Remember that people like Moxie Marlinspike already boycott fdroid bcause they do not want to encourage people to download from untrusted sources. If we encourage the srarching of apps outside our control, he will have an even stronger point. So no, please don't...

Thanks for your comment, Sebastian, you got me wondering why the argument that "it allows users to download random apps" comes up again. So learned that the F-Droid client installs apps from the internet/local memory/...

I always thought expected, it would install only apps it downloaded itself.

My fault, could have checked that before -.-

Given that, I agree with all of you that this feature should not be included in its current form.

You may ignore my above statements regarding this argument and those of its kind...

Yet, I think the use-cases I described earlier remain. How would you think about it if we could get a user from concluding: "F-Droid told me to search the APK in the internet and installed it without complaining, so it must be save." to "F-Droid told me I could search the internet but it also explicitly told me it is not save and I am on my own with this one."?

E.g. the whole management thing could be moved into the settings activity (where it belongs...). Additionally, I would add a checkbox that enables/disables the feature and show an informative text when it is enabled and/or when the user clicks on a search list entry.

The information should contain something like:

• if you install untrustworthy apps, they might leak or destroy your personal data
• apps you find in the internet might be non-free
• you will have to take the full responsibility for your actions
• ...?

At least there would be a word of warning unlike when you attempt to install an apk from file. Or is there one and I missed it again?!

However, I also agree that this would still add another point on the list of those who argue against F-Droid, since it offers yet another way how to find malicious or non-free apps. Personally, I would rather show a warning and try to raise an awareness of security implications than trying to "protect" users give users the impression they are "protected" from all bad that can happen to them in the wild wild web (not possible to do so anyway, imho)

Edit: latter part of last sentence was ambiguous, rephrased it.

Hey @fmuenchbach, thanks a lot for what you've done here. Not only the code, but also the detailed MR and screenshots which help to understand what it is you've contributed. To be honest, I don't recall us having any contentious features submitted from the community as of yet, so please don't be offended if there is a lot of discussion about whether this is a good idea or not, or what the final outcome is.

As for my personal opinion, I do understand the security concerns that others have raised. I'm less convinced by the free software argument, because the F-Droid client can very much be used to distribute proprietary software if somebody so chooses to do that. There is perhaps a slight confusion between the "fdroidclient" part of the F-Droid project and the "fdroiddata" part (which builds open source apks for the https://f-droid.org/repo repository).

My opinion is not about either of these, but rather about using the right tool for the job. I experienced this in the past when adding a "Run" button to the app details screen. I had to look twice to make sure that I wasn't using F-Droid as a full-blown-launcher-application - many of these exist and do the job really well, so F-Droid probably shouldn't do that.

The same can probably be said for search. The Android system already has a mechanism for searching arbitrary applications, which most browsers implement. To see Intent.ACTION_SEARCH in action, run this command:

adb shell am start -a android.intent.action.SEARCH -e "query" "BLAH"

It will ask which application you want to open to complete the search. I understand this is not quite the same as what you have implemented whereby users can specify their own search providers. However users should have already configured their browsers with their favourite search engines. Thus, sending a search intent to the browser should search using that engine. Again, it is not perfect - nobody will have the Play/Amazon stores configured as their default search providers. But again, those two apps will likely listen for the SEARCH intent as well.

So could you please comment on this proposal:

When a search fails, show a message saying the search failed in the SearchActivity, and add a means for the user to "Try search using another app" or something along those lines. This would fire up an Intent.ACTION_SEARCH intent.

Hi!

sorry, I did not find the time to respond earlier.

Thanks to @pserwylo for encouraging me. I am not offended by the discussion, I would rather be offended if there would be no discussion about a MR coming out of the blue like this one.

Basically, I do like the "one-button"-solution you proposed, but as you also said its not quite the same thing. I wanted to keep it as generic as possible to allow the users to freely configure what app / search engine they want to use. However, I also do see arguments against this approach since it is more complex (to maintain and to use) and it indeed is not the job of fdroidclient to act as the user's "search hub".

Assuming there is a will to implement a way to forward search queries to another app / search engine, I think, we should first try to figure out what additional functionality it should provide. The questions I have in mind are:

• How should the user be able to select the destination of the forwarded query? Only the user's default search engine? Only web searches and play? Arbitrary sites?
• Does a user want to be able to choose different destinations each time he uses the feature?
• Should we also address use cases where the users are able to choose different destinations each time they use the feature?

This, in turn, would also help to decide how to implement it and which Android Intent to use.

About the Intents in general:

Disclaimer: I'm for sure not the most sophisticated android developer, so handle with care.

android.intent.ACTION_SEARCH is handled by any app offering a search. I just tested it on a friend's device with Play installed. Beside of the browser and Play it also offered the email app, mediaplayers, the address book and basically anything else with a search. In this particular case the list was long enough to scroll twice and. I expect this to be very confusing for a user when he clicks on "Try search using another app" and the address book pops up as one of the first entries in the list...

android.intent.ACTION_WEB_SEARCH apears to be more what we need here, but it does not open Play. Yet, the API docs say

Activity Action: Perform a web search.

Input: getStringExtra(SearchManager.QUERY) is the text to search for. If it is a url starts with http or https, the site will be opened. If it is plain text, Google search will be applied.

On one hand, this would make it somehow flexible since it offers a way to configure another search engine / app (not direct, it still requires a little piece code to set the query). On the other hand, it always opens Google by default no matter what you configured in your browser.

For those who want to try it out, just use (with or without -d, doesn't matter):

adb shell am start -a android.intent.action.WEB_SEARCH "http://gitlab.com"

android.intent.ACTION_VIEW: (thats what I used in my code) Is very flexible, since it is very generic. But in order to make real use of it, it requires quite a lot of code, providing management functionality plus more UI integration. For http[s] links the list of apps contains mostly browsers. Play is only listed for "play-links" (starting with http[s]://play.google.com/store).

From what others have said before, I expect a single button solution to gain wider approval than my current attempt, since it would neither need a real "Manager" activity nor transforming the client into a search hub. A good trade-off between simplicity and adaptability here could be to use one single button and add an entry to the settings letting the user exchange the (one!) destination. Using ACTION_VIEW and replacing a certain pattern within the link would still allow the user to direct the query wherever he wants.

PS: When I tried ACTION_SEARCH the list I got looked somehow weird. It had a lot of duplicated entries and a lot of entries in the form of com.android.vending (Play Store App) or com.android.browser (stock browser) as entry name. Can be reproduced on my friends stock Samsung android (4.0) phone as well as on my cm android (4.2.2). Anyone knows why?

Hey @fmuenchbach - I'm going to look at these intents over the next week to check out what you've mentioned here.

In the mean time, I had been working on improving the search in MR !177 (merged) which you may be interest in. It does away with the SearchResults activity completely, and applies the search on the current view. While this does break the initial proposal you had here (with the 'External" tab in search results), it should not preclude another solution such as showing a specific UI instead of the empty results list which triggers an external search in another app/site.

Status changed to closed