DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is responsive to the amendments and remarks received 23 July 2026. Claims 1 - 6 and 9 - 20 are currently pending.
Specification
The objection to the disclosure, due to containing an embedded hyperlink and/or other form of browser-executable code, is hereby withdrawn in view of the amendments and remarks received 23 July 2026.
Claim Objections
The objections to claims 10, 11, 15 and 17, due to minor informalities, are hereby withdrawn in view of the amendments and remarks received 23 July 2026.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
The rejections to claims 11 - 20 under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, are hereby withdrawn in view of the amendments and remarks received 23 July 2026.
Claim Rejections - 35 USC § 101
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The rejections to claims 1, 2, 5 - 7, 9 - 13, 15 and 16 under 35 U.S.C. 101 are hereby withdrawn in view of the amendments and remarks received 23 July 2026.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 - 6 and 9 - 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments filed 23 July 2026 have been fully considered but they are not persuasive.
On pages 13 - 14 of the remarks the Applicant’s Representative argues that the previously cited references fail to teach or anticipate “determining to perform a dedicated diagnostic processing flow instead of general search engine processing based on the diagnostic search intent, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet; in response to determining a dedicated diagnostic processing flow based on the diagnostic search intent, providing the one or more images to a medical conditions classification model based on the intent classification”. In particular, the Applicant’s Representative argues that Lyman et al. fail to teach or disclose the aforementioned disputed claim limitation(s) at least because Lyman et al. fail to teach “an intent classification model or a diagnostic search intent determination, let alone leveraging the intent classification to determine a diagnostic processing flow is to be performed.” Therefore, the Applicant’s Representative argues that the previously cited references fail to teach or disclose the aforementioned disputed claim limitation(s).
The Examiner respectfully disagrees, in part.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Additionally, Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references.
Furthermore, the Examiner asserts that, at least, Lyman et al. disclose determining to perform a dedicated diagnostic processing flow based on the diagnostic search intent and in response to determining a dedicated diagnostic processing flow based on the diagnostic search intent, providing the one or more images to a medical conditions classification model based on the intent classification, see at least page 46 paragraph 0323 - page 47 paragraph 0331 and page 49 paragraphs 0341 - 0342 of Lyman et al. wherein it is disclosed that “the captured image processing system 4004 first performs a image classifier function 4007, for example, to determine an anatomical region of the image, to determine anatomical features in the image that need to be diagnosed, and/or to otherwise classify the image into one of a plurality of image types. Once the image type is determined by performing the image classifier function 4007 on the captured image, one of a plurality of different processing functions 4008, each corresponding to one of the plurality of image types, can be selected for performance on the image data” [0325] (emphasis added), that “captured images of physical, anatomical features of a patient, taken by a camera, can be similarly be processed by the captured image processing system to generate diagnosis data. For example, a patient can utilize a camera on their mobile device to take a picture of a concerning abnormality on their skin and/or an externally observable abnormality that is otherwise visually observable via the human eye and/or via the image capture device 4012 communicating and/or coupled to their client device” [0328], that “image classifier function 4007 can detect the anatomical region, type of bodily tissue and/or bodily fluid captured, and/or other classifying information for the captured image to determine the processing function 4008” [0329], that “Each processing function 4008 can utilize a computer vision model trained to diagnose conditions of the corresponding type of image” [0330] and that “multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis” [0331]. (emphasis added)
The Examiner asserts that, as shown herein above and in the cited portions, Lyman et al. disclose that in response to determining an anatomical region and/or features in an image that need to be diagnosed, the image is provided to one or more particular processing functions that each utilize a trained computer vision model for performance, i.e., determining to perform a dedicated diagnostic processing flow based on the diagnostic search intent and in response to determining a dedicated diagnostic processing flow based on the diagnostic search intent, providing the one or more images to a medical conditions classification model based on the intent classification. The Examiner asserts that, at least, image classifier function 4007 and/or first processing function 4008 trained to detect skin conditions of Lyman et al. correspond to an intent classification model and that, at least, the process(es) of determining anatomical features in an image that need to be diagnosed and/or determining that an image is a picture of skin with a rash of Lyman et al. correspond to a diagnostic search intent determination.
The Examiner notes that Peng et al. in view of Lyman et al. do not appear to explicitly disclose performing the dedicated processing flow instead of a general search engine processing, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet. However, the Examiner asserts that newly applied analogous prior art Regan et al. disclose the aforementioned limitation(s) missing from Peng et al. in view of Lyman et al., see the analysis of instant claims 1, 11 and 17 included herein below in section 13 of the instant Office Action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1 - 6, 9 and 11 - 20 are rejected under 35 U.S.C. 103 as being unpatentable over Peng et al. U.S. Publication No. 2021/0019342 A1 in view of Lyman et al. U.S. Publication No. 2020/0160980 A1 in view of Regan et al. U.S. Publication No. 2013/0019202 A1.
- With regards to claim 1, Peng et al. disclose a computing system for medical condition visual search, (Peng et al., Abstract, Figs. 1, 8 - 11 & 17, Pg. 1 ¶ 0001 and 0003, Pg. 3 ¶ 0040, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 - 0065) the system comprising: one or more processors; (Peng et al., Abstract, Fig. 8, Pg. 1 ¶ 0003, Pg. 2 ¶ 0016, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 and 0065, Pg. 8 ¶ 0088, Pg. 9 ¶ 0105 [“computer system”]) and one or more instructions that, when executed by the one or more processors, cause the computing system to perform operations, (Peng et al., Abstract, Fig. 8, Pg. 1 ¶ 0003, Pg. 2 ¶ 0016, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 and 0065, Pg. 8 ¶ 0088, Pg. 9 ¶ 0105 [“computer system”]) the operations comprising: obtaining a search query, wherein the search query comprises one or more images, wherein the one or more images depict a body part of a user; (Peng et al., Abstract, Figs. 1, 2, 9 - 12 & 17, Pg. 1 ¶ 0003 - 0004, Pg. 3 ¶ 0040 - 0042, Pg. 4 ¶ 0049 - 0051, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0071, Pg. 8 ¶ 0092 - 0099, Pg. 9 ¶ 0102 - 0105) wherein the search query has a diagnostic search intent; (Peng et al., Figs. 1, 2, 10, 11, 13 & 14, Pg. 1 ¶ 0008 and 0011 - 0012, Pg. 2 ¶ 0017 - 0020, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050 - 0051, Pg. 5 ¶ 0054, Pg. 6 ¶ 0062 - Pg. 7 ¶ 0068, Pg. 7 ¶ 0070, Pg. 8 ¶ 0089 - 0091, 0093 and 0096 - 0099, Pg. 9 ¶ 0102 and 0110) providing the one or more images to a medical conditions classification model based on the diagnostic search intent; (Peng et al., Abstract, Figs. 6 - 11, Pg. 1 ¶ 0003 - 0005, Pg. 2 ¶ 0016 - 0020, Pg. 3 ¶ 0045 - 0047, Pg. 4 ¶ 0049 - 0050, Pg. 5 ¶ 0054 - 0059, Pg. 6 ¶ 0061 - 0063 and 0065, Pg. 7 ¶ 0083 - Pg. 8 ¶ 0086, Pg. 8 ¶ 0088) processing the one or more images with the medical conditions classification model to generate one or more predicted condition classifications, (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0008 and 0012, Pg. 2 ¶ 0016 - 0020, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050, Pg. 5 ¶ 0054, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0070, Pg. 8 ¶ 0088 - 0091 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100) wherein the one or more predicted condition classifications are descriptive of one or more candidate medical conditions determined to be potentially depicted in the one or more images; (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0008 and 0012, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050, Pg. 6 ¶ 0063 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0088 - 0091 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100) and providing medical condition information associated with the one or more candidate medical conditions for display in a search results interface, (Peng et al., Figs. 1, 2, 10 - 12 & 17, Pg. 1 ¶ 0006 - 0008 and 0011 - 0012, Pg. 2 ¶ 0017 - 0022, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0086 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100, Pg. 9 ¶ 0102 - 0110) wherein the search results interface comprises: a first panel comprising the medical condition information; (Peng et al., Figs. 1, 2, 10 & 11, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0063 - 0065, Pg. 8 ¶ 0094 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100) and a second panel comprising a plurality of visual search results, (Peng et al., Figs. 1, 2, 10, 11 & 17, Pg. 1 ¶ 0006 - 0008 and 0011, Pg. 2 ¶ 0022 - 0024 and 0032 - 0033, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 - 0065, Pg. 8 ¶ 0086 - 0087 and 0094 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100, Pg. 9 ¶ 0102 and 0110) wherein the plurality of visual search results are determined based on a determined visual similarity with the one or more images. (Peng et al., Abstract, Figs. 6 & 17, Pg. 1 ¶ 0003 and 0005 - 0007, Pg. 4 ¶ 0048 - 0050, Pg. 5 ¶ 0052 - 0055 and 0059, Pg. 6 ¶ 0061) Peng et al. fail to disclose explicitly one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations; processing the one or more images with an intent classification model to generate an intent classification, wherein the intent classification indicates that the search query has a diagnostic search intent, wherein the intent classification model was trained to determine a search intent of the user based on one or more features in an input image; determining to perform a dedicated diagnostic processing flow instead of general search engine processing based on the diagnostic search intent, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet; and in response to determining a dedicated diagnostic processing flow based on the diagnostic search intent, providing the images to the medical conditions classification model based on the intent classification. Pertaining to analogous art, Lyman et al. disclose a computing system for medical condition visual search, (Lyman et al., Figs. 1 - 3 & 15A - 16A, Pg. 1 ¶ 0034 - Pg. 2 ¶ 0039, Pg. 2 ¶ 0041, Pg. 4 ¶ 0047 - 0050, Pg. 4 ¶ 0053 - Pg. 5 ¶ 0056, Pg. 46 ¶ 0323 - 0327, Pg. 49 ¶ 0339 - 0342, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) the system comprising: one or more processors; (Lyman et al., Pg. 1 ¶ 0034 - Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, (Lyman et al., Pg. 1 ¶ 0034 - Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) the operations comprising: obtaining a search query, wherein the search query comprises one or more images, wherein the one or more images depict a body part of a user; (Lyman et al., Figs. 1 - 3 & 15A - 16A, Pg. 14 ¶ 0108 - 0110, Pg. 46 ¶ 0323 - Pg. 47 ¶ 0332, Pg. 48 ¶ 0334 - 0336, Pg. 49 ¶ 0340 - 0342) processing the one or more images with an intent classification model to generate an intent classification, (Lyman et al., Pg. 46 ¶ 0324 - 0325, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“Each processing function 4008 can utilize a computer vision model trained to diagnose conditions of the corresponding type of image, for example, where each processing function 4008 is trained on a training set of images of the corresponding type as input data and on corresponding diagnosis data as output labels” and “multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) wherein the intent classification indicates that the search query has a diagnostic search intent, (Lyman et al., Pg. 46 ¶ 0324 - 0325, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“Each processing function 4008 can utilize a computer vision model trained to diagnose conditions of the corresponding type of image, for example, where each processing function 4008 is trained on a training set of images of the corresponding type as input data and on corresponding diagnosis data as output labels” and “multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) wherein the intent classification model was trained to determine a search intent of the user based on one or more features in an input image; (Lyman et al., Pg. 46 ¶ 0324 - 0325, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“Each processing function 4008 can utilize a computer vision model trained to diagnose conditions of the corresponding type of image, for example, where each processing function 4008 is trained on a training set of images of the corresponding type as input data and on corresponding diagnosis data as output labels”]) determining to perform a dedicated diagnostic processing flow based on the diagnostic search intent; (Lyman et al., Pg. 46 ¶ 0324 - Pg. 47 ¶ 0331, Pg. 49 ¶ 0341 - 0342 [“the captured image processing system 4004 first performs a image classifier function 4007, for example, to determine an anatomical region of the image, to determine anatomical features in the image that need to be diagnosed, and/or to otherwise classify the image into one of a plurality of image types. Once the image type is determined by performing the image classifier function 4007 on the captured image, one of a plurality of different processing functions 4008, each corresponding to one of the plurality of image types, can be selected for performance on the image data”, “image classifier function 4007 can detect the anatomical region, type of bodily tissue and/or bodily fluid captured, and/or other classifying information for the captured image to determine the processing function 4008” and “multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) in response to determining a dedicated diagnostic processing flow based on the diagnostic search intent, providing the one or more images to a medical conditions classification model based on the intent classification; (Lyman et al., Pg. 46 ¶ 0324 - Pg. 47 ¶ 0331, Pg. 49 ¶ 0341 - 0342 [“multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) processing the one or more images with the medical conditions classification model to generate one or more predicted condition classifications, (Lyman et al., Figs. 15A - 15C, Pg. 8 ¶ 0070 - 0075, Pg. 46 ¶ 0323 - 0326, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342, Pg. 51 ¶ 0350 - 0353) wherein the one or more predicted condition classifications are descriptive of one or more candidate medical conditions determined to be potentially depicted in the one or more images; (Lyman et al., Figs. 15A - 15C, Pg. 8 ¶ 0070 - 0075, Pg. 46 ¶ 0323 - 0326, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342, Pg. 51 ¶ 0350 - 0353) and providing medical condition information associated with the one or more candidate medical conditions. (Lyman et al., Figs. 15A - 15C, Pg. 8 ¶ 0070 - 0075, Pg. 46 ¶ 0323 - 0326, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342, Pg. 51 ¶ 0350 - 0353) Lyman et al. fail to disclose explicitly performing the dedicated processing flow instead of general search engine processing, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet. Pertaining to analogous art, Regan et al. disclose obtaining a search query; (Regan et al., Abstract, Figs. 2, 3, 8 & 12, Pg. 1 ¶ 0011 - 0013, Pg. 3 ¶ 0046, Pg. 4 ¶ 0054 - Pg. 5 ¶ 0055, Pg. 5 ¶ 0059 - 0061) processing the search query to generate an intent classification, (Regan et al., Figs. 2 & 12, Pg. 3 ¶ 0046, Pg. 4 ¶ 0049, Pg. 4 ¶ 0054 - Pg. 5 ¶ 0055, Pg. 9 ¶ 0096 - 0101) wherein the intent classification indicates that the search query has a search intent; (Regan et al., Figs. 2 & 12, Pg. 3 ¶ 0046, Pg. 4 ¶ 0049, Pg. 4 ¶ 0054 - Pg. 5 ¶ 0055, Pg. 9 ¶ 0096 - 0101, Pg. 10 ¶ 0105 - 0108, Pg. 19 ¶ 0111 - Pg. 20 ¶ 0114) and determining to perform a dedicated diagnostic processing flow instead of general search engine processing based on the search intent, (Regan et al., Figs. 2, 8 & 12, Pg. 1 ¶ 0009 - 0013, Pg. 3 ¶ 0046, Pg. 4 ¶ 0049 - 0050, Pg. 10 ¶ 0105 - Pg. 19 ¶ 0111, Pg. 20 ¶ 0114 - 0115) wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet. (Regan et al., Figs. 2, 3, 8 & 9, Pg. 1 ¶ 0003 - 0009, Pg. 20 ¶ 0115) Peng et al. and Lyman et al. are combinable because they are both directed towards medical image processing systems that process dermatology images depicting skin to generate predicted diagnosis data for the dermatology images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Peng et al. with the teachings of Lyman et al. A first modification would have been prompted in order to enhance the base device of Peng et al. with the well-known and applicable technique Lyman et al. applied to a comparable device. Utilizing one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more processors, cause a computing system to perform operations, as taught by Lyman et al., to implement operations of the base device of Peng et al. would enhance the base device of Peng et al. by ensuring that its operations are carried out accurately and efficiently at high-computational speed on computer architecture, by facilitating widespread distribution of the base device of Peng et al. to millions of potential end-users with access to a computer and by simplifying the process of making revisions, modifications and/or updates to the operations of the base device of Peng et al. A second modification would have been prompted in order to enhance the base device of Peng et al. with the well-known and applicable technique Lyman et al. applied to a comparable device. Processing the one or more images with an intent classification model to generate an intent classification and providing the one or more images to the medical conditions classification model based on the intent classification, as taught by Lyman et al., would enhance the base device of Peng et al. by reducing the total number of image processing operations it performs since only search query images determined to have a diagnostic search intent would undergo processing by the medical conditions classification model thereby eliminating unnecessary image processing operations from being performed by the medical conditions classification model on search query images lacking diagnostic search intent and improving the overall computational efficiency of the base device of Peng et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more processors, cause a computing system to perform operations would be utilized to implement the base device of Peng et al. so as to ensure that its operations are carried out accurately and efficiently at high-computational speed on computer architecture, to facilitate widespread distribution of the base device of Peng et al. to millions of potential end-users with access to a computer and to simplify the process of making revisions, modifications and/or updates to the operations of the base device of Peng et al. and in that the one or more images would be processed with an intent classification model to generate an intent classification and provided to the medical conditions classification model based on the intent classification so as to help ensure that unnecessary image processing operations are not performed by the medical conditions classification model on search query images lacking a diagnostic search intent and thereby improve the overall computational efficiency of the base device of Peng et al. In addition, Peng et al. in view of Lyman et al. and Regan et al. are combinable because they are all directed towards systems and methods that process search queries of users to return the most pertinent and relevant search results possible. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Peng et al. in view of Lyman et al. with the teachings of Regan et al. This modification would have been prompted in order to enhance the combined base device of Peng et al. in view of Lyman et al. with the well-known and applicable technique of Regan et al. Performing the dedicated processing flow instead of general search engine processing based on the search intent, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet, as taught by Regan et al., would enhance the combined base device by helping ensure that users are quickly and efficiently provided with the most relevant information possible in response to their search queries since in cases where the intent of their search queries cannot be definitively determined general search engine processing would instead be performed and/or by allowing for users to perform searches for an increased number and/or variety of different types of information thereby improving the overall appeal, usefulness and marketability of the combined base device to potential end-users. Furthermore, this modification would have been prompted by the teachings and suggestions of Peng et al. that tools for searching for medical images include commercially available products or services such as Google’s Image Search function, that the reference library they search can be generated from a variety of open sources, that the reference library can be contained on remote server, that a user is given a choice of databases to search, and that search results can be filtered by image source and/or various metadata associated with the images, see at least figures 3 and 8 - 10, page 1 paragraph 0002, page 4 paragraph 0051 and page 6 paragraphs 0061 - 0063 of Peng et al. Moreover, this modification would have been prompted by the teachings and suggestions of Lyman et al. that captured image and/or diagnosis data can alternatively or additionally be transmitted to any other medical entity such as a doctor of the patient and/or to another client device to generate additional diagnosis data, and that in some embodiments a single processing function 4008 is performed on all captured images and/or no image classifier function 4007 is performed, see at least page 46 paragraph 0326 and page 47 paragraph 0331 of Lyman et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that in situations wherein a search query is not indicated as having a diagnostic search intent the combined base device would perform general search engine processing instead of the dedicated diagnostic processing flow so as to help ensure that users are quickly and efficiently provided with the most relevant and pertinent information possible in response to their search queries. Therefore, it would have been obvious to combine Peng et al. with Lyman et al. and Regan et al. to obtain the invention as specified in claim 1.
- With regards to claim 2, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the system of claim 1. Peng et al. fail to disclose explicitly processing the one or more images with a skin classification model to determine the one or more images depict skin, wherein the skin classification model was trained to determine whether an input image depicts skin; and providing the one or more images to the intent classification model based on the one or more images depicting skin. Pertaining to analogous art, Lyman et al. disclose processing the one or more images with a skin classification model to determine the one or more images depict skin, (Lyman et al., Pg. 46 ¶ 0323 - 0327, Pg. 47 ¶ 0329 - 0331, Pg. 49 ¶ 0341 - 0343) wherein the skin classification model was trained to determine whether an input image depicts skin; (Lyman et al., Pg. 46 ¶ 0323 - 0327, Pg. 47 ¶ 0329 - 0331, Pg. 49 ¶ 0341 - 0343) and providing the one or more images to the intent classification model based on the one or more images depicting skin. (Lyman et al., Pg. 47 ¶ 0329 - 0331, Pg. 49 ¶ 0342 [“multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis.”])
- With regards to claim 3, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the system of claim 1, wherein the operations further comprise: obtaining the medical condition information associated with the one or more candidate medical conditions from a curated medical information database, (Peng et al., Figs. 3, 8 & 9, Pg. 1 ¶ 0012 - 0013, Pg. 3 ¶ 0046 - 0047, Pg. 4 ¶ 0051, Pg. 6 ¶ 0062 - 0065) wherein the medical condition information comprises a medical condition name and one or more condition images, (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0006 - 0008 and 0011 - 0013, Pg. 3 ¶ 0040 - 0044 and 0046 - 0047, Pg. 4 ¶ 0050 - 0051, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0070, Pg. 8 ¶ 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102) wherein the one or more condition images depict an example of the respective candidate medical condition. (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0006 - 0008 and 0011 - 0013, Pg. 3 ¶ 0040 - 0044 and 0046 - 0047, Pg. 4 ¶ 0050 - 0051, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0070, Pg. 8 ¶ 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102)
- With regards to claim 4, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the system of claim 3, wherein the one or more condition images are obtained from a medical condition image database, (Peng et al., Figs. 3, 8 & 9, Pg. 1 ¶ 0012 - 0013, Pg. 3 ¶ 0046 - 0047, Pg. 4 ¶ 0051, Pg. 6 ¶ 0062 - 0065) wherein the medical condition image database comprises a plurality of medical condition images selected by one or more medical professionals. (Peng et al., Figs. 3, 8 & 9, Pg. 1 ¶ 0012 - 0013, Pg. 2 ¶ 0025, Pg. 3 ¶ 0046 - 0047, Pg. 4 ¶ 0051, Pg. 6 ¶ 0062 - Pg. 7 ¶ 0066, Pg. 7 ¶ 0069 - 0070, Pg. 8 ¶ 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102)
- With regards to claim 5, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the system of claim 1, wherein the operations further comprise: processing the one or more images to determine a region of interest; (Peng et al., Figs. 1 & 2, Pg. 1 ¶ 0006, Pg. 2 ¶ 0017 - 0018, Pg. 3 ¶ 0042 - 0043, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0062 and 0065, Pg. 8 ¶ 0089 - 0090 and 0099, Pg. 9 ¶ 0104 - 0106) cropping the one or more images to generate one or more cropped images based on the region of interest; (Peng et al., Figs. 1 & 2, Pg. 1 ¶ 0006, Pg. 2 ¶ 0017 - 0018, Pg. 3 ¶ 0042 - 0043, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0062 and 0065, Pg. 8 ¶ 0089 - 0090 and 0099, Pg. 9 ¶ 0104 - 0106) and wherein the one or more cropped images are processed with the medical conditions classification model. (Peng et al., Pg. 1 ¶ 0006, Pg. 3 ¶ 0042 - 0043, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0062, Pg. 8 ¶ 0099, Pg. 9 ¶ 0104 - 0105) Peng et al. fail to disclose explicitly wherein the one or more cropped images are processed with the intent classification model. Pertaining to analogous art, Lyman et al. disclose processing the one or more images to determine a region of interest; (Lyman et al., Pg. 20 ¶ 0141, Pg. 21 ¶ 0143 - 0145, Pg. 46 ¶ 0325, Pg. 47 ¶ 0329 - 0331, Pg. 49 ¶ 0341 - 0343, Pg. 51 ¶ 0353 - 0355, Pg. 52 ¶ 0361 - Pg. 53 ¶ 0362, Pg. 54 ¶ 0375) cropping the one or more images to generate one or more cropped images based on the region of interest; (Lyman et al., Pg. 20 ¶ 0141, Pg. 21 ¶ 0143 - 0145, Pg. 46 ¶ 0325, Pg. 47 ¶ 0329 - 0331, Pg. 49 ¶ 0341 - 0343, Pg. 51 ¶ 0353 - 0355, Pg. 52 ¶ 0361 - Pg. 53 ¶ 0362, Pg. 54 ¶ 0375) and wherein the one or more cropped images are processed with the intent classification model and the medical conditions classification model. (Lyman et al., Pg. 20 ¶ 0140 - Pg. 21 ¶ 0145, Pg. 46 ¶ 0325, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0340 - 0342, Pg. 51 ¶ 0353 [“the captured image is pre-processed with filters to change brightness, contrast, orientation, and/or to crop the image before the image classifier function 4007 and/or processing functions 4008 is applied.”])
- With regards to claim 6, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the system of claim 1, wherein the operations further comprise: processing the one or more images to determine a region of interest; (Peng et al., Figs. 1 & 2, Pg. 1 ¶ 0006, Pg. 2 ¶ 0017 - 0018, Pg. 3 ¶ 0042 - 0043, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0062 and 0065, Pg. 8 ¶ 0089 - 0090 and 0099, Pg. 9 ¶ 0104 - 0106) generating an annotated image based on the one or more images and the region of interest, (Peng et al., Figs. 1, 2 & 10, Pg. 1 ¶ 0006 and 0011 - 0012, Pg. 2 ¶ 0017 - 0020, Pg. 3 ¶ 0042 - 0044 and 0047, Pg. 4 ¶ 0050, Pg. 5 ¶ 0054, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0089 - 0092 and 0096, Pg. 9 ¶ 0105 - 0110) wherein the annotated image comprises the one or more images with one or more indicators, (Peng et al., Figs. 1 & 2, Pg. 1 ¶ 0006 and 0011 - 0012, Pg. 2 ¶ 0017 - 0020, Pg. 3 ¶ 0042 - 0044, Pg. 4 ¶ 0050, Pg. 5 ¶ 0054, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0089 - 0092 and 0096, Pg. 9 ¶ 0105 - 0110) wherein the one or more indicators indicate a location of the region of interest in the one or more images; (Peng et al., Figs. 1 & 2, Pg. 1 ¶ 0006, Pg. 2 ¶ 0017 - 0020, Pg. 3 ¶ 0042 - 0043, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0062 - 0065, Pg. 8 ¶ 0089 - 0092, 0096 and 0099, Pg. 9 ¶ 0105 - 0110) and providing the annotated image for display with the medical condition information as an output. (Peng et al., Figs.1, 2, 10, 11 & 17, Pg. 1 ¶ 0003, 0006 - 0009 and 0011 - 0012, Pg. 2 ¶ 0017 - 0022, Pg. 3 ¶ 0042 - 0044, Pg. 4 ¶ 0049 - 0050, Pg. 5 ¶ 0054, Pg. 6 ¶ 0063 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0089 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100, Pg. 9 ¶ 0104 - 0110) In addition, analogous art Lyman et al. disclose processing the one or more images to determine a region of interest; (Lyman et al., Pg. 20 ¶ 0141, Pg. 21 ¶ 0143 - 0145, Pg. 46 ¶ 0325, Pg. 47 ¶ 0329 - 0331, Pg. 49 ¶ 0341 - 0343, Pg. 51 ¶ 0353 - 0355, Pg. 52 ¶ 0361 - Pg. 53 ¶ 0362, Pg. 54 ¶ 0375) generating an annotated image based on the one or more images and the region of interest, (Lyman et al., Fig. 16D, Pg. 51 ¶ 0352 - 0353, Pg. 54 ¶ 0375) wherein the annotated image comprises the one or more images with one or more indicators, (Lyman et al., Fig. 16D, Pg. 51 ¶ 0352 - 0353, Pg. 54 ¶ 0375) wherein the one or more indicators indicate a location of the region of interest in the one or more images; (Lyman et al., Fig. 16D, Pg. 51 ¶ 0352 - 0353, Pg. 54 ¶ 0375) and providing the annotated image for display with the medical condition information as an output. (Lyman et al., Fig. 16D, Pg. 51 ¶ 0352 - 0353, Pg. 54 ¶ 0375)
- With regards to claim 9, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the system of claim 1, wherein the search results interface comprises: a selectable user interface element, wherein the selectable user interface element is associated with a particular medical condition of the one or more candidate medical conditions, (Peng et al., Figs. 1, 10 & 17, Pg. 1 ¶ 0009, Pg. 4 ¶ 0049, Pg. 6 ¶ 0063 - 0064, Pg. 8 ¶ 0096 and 0099, Pg. 9 ¶ 0110) and wherein the selectable user interface element is provided adjacent to the medical condition information. (Peng et al., Figs. 1, 10 & 17, Pg. 1 ¶ 0008 - 0012, Pg. 4 ¶ 0048 - 0049, Pg. 6 ¶ 0061 - 0064, Pg. 8 ¶ 0086 - 0087, 0094 - 0096 and 0099)
- With regards to claim 11, Peng et al. disclose a computer-implemented method for skin condition search, (Peng et al., Abstract, Figs. 1, 8 - 11 & 17, Pg. 1 ¶ 0001 and 0003 - 0004, Pg. 3 ¶ 0040 - 0044, Pg. 4 ¶ 0049 - 0051, Pg. 6 ¶ 0061 - 0065, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102, Pg. 9 ¶ 0110) the method comprising: obtaining, by a computing system comprising one or more processors, (Peng et al., Abstract, Fig. 8, Pg. 1 ¶ 0003, Pg. 2 ¶ 0016, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 and 0065, Pg. 8 ¶ 0088, Pg. 9 ¶ 0105 [“computer system”]) a search query, wherein the search query comprises one or more images; (Peng et al., Abstract, Figs. 1, 2, 9 - 12 & 17, Pg. 1 ¶ 0003 - 0004, Pg. 3 ¶ 0040 - 0042, Pg. 4 ¶ 0049 - 0051, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0071, Pg. 8 ¶ 0092 - 0099, Pg. 9 ¶ 0102 - 0105) wherein the search query has a diagnostic search intent; (Peng et al., Figs. 1, 2, 10, 11, 13 & 14, Pg. 1 ¶ 0008 and 0011 - 0012, Pg. 2 ¶ 0017 - 0020, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050 - 0051, Pg. 5 ¶ 0054, Pg. 6 ¶ 0062 - Pg. 7 ¶ 0068, Pg. 7 ¶ 0070, Pg. 8 ¶ 0089 - 0091, 0093 and 0096 - 0099, Pg. 9 ¶ 0102 and 0110) providing, by the computing system, (Peng et al., Abstract, Fig. 8, Pg. 1 ¶ 0003, Pg. 2 ¶ 0016, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 and 0065, Pg. 8 ¶ 0088, Pg. 9 ¶ 0105 [“computer system”]) the one or more images to a dermatology conditions classification model based on the diagnostic search intent; (Peng et al., Abstract, Figs. 6 - 11, Pg. 1 ¶ 0003 - 0005, Pg. 2 ¶ 0016 - 0020, Pg. 3 ¶ 0045 - 0047, Pg. 4 ¶ 0049 - 0050, Pg. 5 ¶ 0054 - 0059, Pg. 6 ¶ 0061 - 0063 and 0065, Pg. 7 ¶ 0083 - Pg. 8 ¶ 0086, Pg. 8 ¶ 0088) processing, by the computing system, (Peng et al., Abstract, Fig. 8, Pg. 1 ¶ 0003, Pg. 2 ¶ 0016, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 and 0065, Pg. 8 ¶ 0088, Pg. 9 ¶ 0105 [“computer system”]) the one or more images with the dermatology conditions classification model to generate one or more predicted condition classifications, (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0008 and 0012, Pg. 2 ¶ 0016 - 0020, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050, Pg. 5 ¶ 0054, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0070, Pg. 8 ¶ 0088 - 0091 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100) wherein the one or more predicted condition classifications are descriptive of one or more candidate skin conditions; (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0004, 0008 and 0012, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050 - 0051, Pg. 6 ¶ 0062 - 0065, Pg. 8 ¶ 0088 - 0091 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102) and providing, by the computing system, (Peng et al., Abstract, Fig. 8, Pg. 1 ¶ 0003, Pg. 2 ¶ 0016, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 and 0065, Pg. 8 ¶ 0088, Pg. 9 ¶ 0105 [“computer system”]) skin condition information associated with the one or more candidate skin conditions as an output for display in a search results interface, (Peng et al., Figs. 1, 2, 10 - 12 & 17, Pg. 1 ¶ 0006 - 0008 and 0011 - 0012, Pg. 2 ¶ 0017 - 0022, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0086 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100, Pg. 9 ¶ 0102 - 0110) wherein the search results interface comprises: a first panel comprising the Skin condition information; (Peng et al., Figs. 1, 2, 10 & 11, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0063 - 0065, Pg. 8 ¶ 0094 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100) and a second panel comprising a plurality of different visual search results, (Peng et al., Figs. 1, 2, 10, 11 & 17, Pg. 1 ¶ 0006 - 0008 and 0011, Pg. 2 ¶ 0022 - 0024 and 0032 - 0033, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 - 0065, Pg. 8 ¶ 0086 - 0087 and 0094 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100, Pg. 9 ¶ 0102 and 0110) wherein the plurality of different visual search results are determined based on a determined visual similarity with the one or more images. (Peng et al., Abstract, Figs. 6 & 17, Pg. 1 ¶ 0003 and 0005 - 0007, Pg. 4 ¶ 0048 - 0050, Pg. 5 ¶ 0052 - 0055 and 0059, Pg. 6 ¶ 0061) Peng et al. fail to disclose explicitly processing the one or more images with a skin classification model to determine the one or more images depict skin; processing the one or more images with an intent classification model to generate an intent classification, wherein the intent classification indicates that the search query has a diagnostic search intent, wherein the intent classification model was trained to determine a search intent of a user based on one or more features in an input image; determining to perform a dedicated diagnostic processing flow instead of general search engine processing based on the diagnostic search intent, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet; and in response to determining a dedicated diagnostic processing flow based on the diagnostic search intent, providing the images to the dermatology conditions classification model based on the intent classification. Pertaining to analogous art, Lyman et al. disclose a computer-implemented method skin condition search, (Lyman et al., Figs. 1 - 3 & 15A - 16A, Pg. 1 ¶ 0034 - Pg. 2 ¶ 0039, Pg. 2 ¶ 0041, Pg. 4 ¶ 0047 - 0050, Pg. 4 ¶ 0053 - Pg. 5 ¶ 0056, Pg. 46 ¶ 0323 - 0327, Pg. 49 ¶ 0339 - 0342, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) the method comprising: obtaining, by a computing system comprising one or more processors, (Lyman et al., Pg. 1 ¶ 0034 - Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) a search query, wherein the search query comprises one or more images; (Lyman et al., Figs. 1 - 3 & 15A - 16A, Pg. 14 ¶ 0108 - 0110, Pg. 46 ¶ 0323 - 0325, Pg. 46 ¶ 0327 - Pg. 47 ¶ 0332, Pg. 48 ¶ 0334 - 0336, Pg. 49 ¶ 0340 - 0342) processing, by the computing system, (Lyman et al., Pg. 1 ¶ 0034 - Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) the one or more images with a skin classification model to determine the one or more images depict skin; (Lyman et al., Pg. 46 ¶ 0323 - 0327, Pg. 47 ¶ 0329 - 0331, Pg. 49 ¶ 0341 - 0343) processing, by the computing system, (Lyman et al., Pg. 1 ¶ 0034 - Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) the one or more images with an intent classification model to generate an intent classification, (Lyman et al., Pg. 46 ¶ 0324 - 0325, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“Each processing function 4008 can utilize a computer vision model trained to diagnose conditions of the corresponding type of image, for example, where each processing function 4008 is trained on a training set of images of the corresponding type as input data and on corresponding diagnosis data as output labels” and “multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) wherein the intent classification indicates that the search query has a diagnostic search intent, (Lyman et al., Pg. 46 ¶ 0324 - 0325, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“Each processing function 4008 can utilize a computer vision model trained to diagnose conditions of the corresponding type of image, for example, where each processing function 4008 is trained on a training set of images of the corresponding type as input data and on corresponding diagnosis data as output labels” and “multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) wherein the intent classification model was trained to determine a search intent of a user based on one or more features in an input image; (Lyman et al., Pg. 46 ¶ 0324 - 0325, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“Each processing function 4008 can utilize a computer vision model trained to diagnose conditions of the corresponding type of image, for example, where each processing function 4008 is trained on a training set of images of the corresponding type as input data and on corresponding diagnosis data as output labels”]) determining, by the computing system, to perform a dedicated diagnostic processing flow based on the diagnostic search intent; (Lyman et al., Pg. 46 ¶ 0324 - Pg. 47 ¶ 0331, Pg. 49 ¶ 0341 - 0342 [“the captured image processing system 4004 first performs a image classifier function 4007, for example, to determine an anatomical region of the image, to determine anatomical features in the image that need to be diagnosed, and/or to otherwise classify the image into one of a plurality of image types. Once the image type is determined by performing the image classifier function 4007 on the captured image, one of a plurality of different processing functions 4008, each corresponding to one of the plurality of image types, can be selected for performance on the image data”, “image classifier function 4007 can detect the anatomical region, type of bodily tissue and/or bodily fluid captured, and/or other classifying information for the captured image to determine the processing function 4008” and “multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) in response to determining a dedicated diagnostic processing flow based on the diagnostic search intent, providing, by the computing system, (Lyman et al., Pg. 1 ¶ 0034 - Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) the one or more images to a dermatology conditions classification model based on the intent classification; (Lyman et al., Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) processing, by the computing system, (Lyman et al., Pg. 1 ¶ 0034 - Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) the one or more images with the dermatology conditions classification model to generate one or more predicted condition classifications, (Lyman et al., Figs. 15A - 15C, Pg. 8 ¶ 0070 - 0075, Pg. 46 ¶ 0323 - 0326, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342, Pg. 51 ¶ 0350 - 0353) wherein the one or more predicted condition classifications are descriptive of one or more candidate skin conditions; (Lyman et al., Figs. 15A - 15C, Pg. 8 ¶ 0070 - 0075, Pg. 46 ¶ 0323 - 0326, Pg. 47 ¶ 0328 - 0332, Pg. 49 ¶ 0341 - 0342, Pg. 51 ¶ 0350 - 0355) and providing, by the computing system, (Lyman et al., Pg. 1 ¶ 0034 - Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) skin condition information associated with the one or more candidate skin conditions as an output. (Lyman et al., Figs. 15A - 15C, Pg. 8 ¶ 0070 - 0075, Pg. 46 ¶ 0323 - 0326, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342, Pg. 51 ¶ 0350 - 0353) Lyman et al. fail to disclose explicitly performing the dedicated processing flow instead of general search engine processing, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet. Pertaining to analogous art, Regan et al. disclose obtaining a search query; (Regan et al., Abstract, Figs. 2, 3, 8 & 12, Pg. 1 ¶ 0011 - 0013, Pg. 3 ¶ 0046, Pg. 4 ¶ 0054 - Pg. 5 ¶ 0055, Pg. 5 ¶ 0059 - 0061) processing the search query to generate an intent classification, (Regan et al., Figs. 2 & 12, Pg. 3 ¶ 0046, Pg. 4 ¶ 0049, Pg. 4 ¶ 0054 - Pg. 5 ¶ 0055, Pg. 9 ¶ 0096 - 0101) wherein the intent classification indicates that the search query has a search intent; (Regan et al., Figs. 2 & 12, Pg. 3 ¶ 0046, Pg. 4 ¶ 0049, Pg. 4 ¶ 0054 - Pg. 5 ¶ 0055, Pg. 9 ¶ 0096 - 0101, Pg. 10 ¶ 0105 - 0108, Pg. 19 ¶ 0111 - Pg. 20 ¶ 0114) and determining to perform a dedicated diagnostic processing flow instead of general search engine processing based on the search intent, (Regan et al., Figs. 2, 8 & 12, Pg. 1 ¶ 0009 - 0013, Pg. 3 ¶ 0046, Pg. 4 ¶ 0049 - 0050, Pg. 10 ¶ 0105 - Pg. 19 ¶ 0111, Pg. 20 ¶ 0114 - 0115) wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet. (Regan et al., Figs. 2, 3, 8 & 9, Pg. 1 ¶ 0003 - 0009, Pg. 20 ¶ 0115) Peng et al. and Lyman et al. are combinable because they are both directed towards medical image processing systems that process dermatology images depicting skin to generate predicted diagnosis data for the dermatology images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Peng et al. with the teachings of Lyman et al. This modification would have been prompted in order to enhance the base device of Peng et al. with the well-known and applicable techniques Lyman et al. applied to a comparable device. Processing the one or more images with a skin classification model to determine the one or more images depict skin, processing the one or more images with an intent classification model to generate an intent classification and providing the one or more images to the dermatology conditions classification model based on the intent classification, as taught by Lyman et al., would enhance the base device of Peng et al. by reducing the total number of image processing operations it performs since only search query images determined to depict skin and have a diagnostic search intent would undergo processing by the dermatology conditions classification model thereby eliminating unnecessary image processing operations from being performed by the dermatology conditions classification model on search query images lacking depictions of skin and/or diagnostic search intent and improving the overall computational efficiency of the base device of Peng et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the one or more images would be processed with a skin classification model to determine the one or more images depict skin, processed with an intent classification model to generate an intent classification and provided to the dermatology conditions classification model based on the intent classification so as to help ensure that unnecessary image processing operations are not performed by the dermatology conditions classification model on search query images lacking depictions of skin and/or a diagnostic search intent and thereby improve the overall computational efficiency of the base device of Peng et al. In addition, Peng et al. in view of Lyman et al. and Regan et al. are combinable because they are all directed towards systems and methods that process search queries of users to return the most pertinent and relevant search results possible. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Peng et al. in view of Lyman et al. with the teachings of Regan et al. This modification would have been prompted in order to enhance the combined base device of Peng et al. in view of Lyman et al. with the well-known and applicable technique of Regan et al. Performing the dedicated processing flow instead of general search engine processing based on the search intent, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet, as taught by Regan et al., would enhance the combined base device by helping ensure that users are quickly and efficiently provided with the most relevant information possible in response to their search queries since in cases where the intent of their search queries cannot be definitively determined general search engine processing would instead be performed and/or by allowing for users to perform searches for an increased number and/or variety of different types of information thereby improving the overall appeal, usefulness and marketability of the combined base device to potential end-users. Furthermore, this modification would have been prompted by the teachings and suggestions of Peng et al. that tools for searching for medical images include commercially available products or services such as Google’s Image Search function, that the reference library they search can be generated from a variety of open sources, that the reference library can be contained on remote server, that a user is given a choice of databases to search, and that search results can be filtered by image source and/or various metadata associated with the images, see at least figures 3 and 8 - 10, page 1 paragraph 0002, page 4 paragraph 0051 and page 6 paragraphs 0061 - 0063 of Peng et al. Moreover, this modification would have been prompted by the teachings and suggestions of Lyman et al. that captured image and/or diagnosis data can alternatively or additionally be transmitted to any other medical entity such as a doctor of the patient and/or to another client device to generate additional diagnosis data, and that in some embodiments a single processing function 4008 is performed on all captured images and/or no image classifier function 4007 is performed, see at least page 46 paragraph 0326 and page 47 paragraph 0331 of Lyman et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that in situations wherein a search query is not indicated as having a diagnostic search intent the combined base device would perform general search engine processing instead of the dedicated diagnostic processing flow so as to help ensure that users are quickly and efficiently provided with the most relevant and pertinent information possible in response to their search queries. Therefore, it would have been obvious to combine Peng et al. with Lyman et al. and Regan et al. to obtain the invention as specified in claim 11.
- With regards to claim 12, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the method of claim 11, further comprising: obtaining, by the computing system, the skin condition information from a curated database, (Peng et al., Figs. 3, 8 & 9, Pg. 1 ¶ 0012 - 0013, Pg. 3 ¶ 0046 - 0047, Pg. 4 ¶ 0051, Pg. 6 ¶ 0062 - 0065) wherein the curated database comprises a plurality of condition datasets associated with a plurality of different skin conditions; (Peng et al., Figs. 3, 8 & 9, Pg. 1 ¶ 0012 - 0013, Pg. 3 ¶ 0040 - 0044 and 0046 - 0047, Pg. 4 ¶ 0049 - 0051, Pg. 6 ¶ 0060 and 0062 - 0065, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102) processing, by the computing system, the one or more images with a search engine to identify the plurality of different visual search results, (Peng et al., Abstract, Figs. 1 - 10 & 17, Pg. 1 ¶ 0003 - 0008 and 0013, Pg. 3 ¶ 0040 - 0044 and 0046, Pg. 4 ¶ 0050, Pg. 5 ¶ 0052 - 0055 and 0059, Pg. 6 ¶ 0061 - 0062, Pg. 8 ¶ 0084 - 0088 and 0094 - 0096) wherein the plurality of different visual search results are associated with a plurality of different web resources; (Peng et al., Figs. 3, 8 & 9, Pg. 1 ¶ 0013, Pg. 3 ¶ 0046 - 0047, Pg. 4 ¶ 0049 - 0051, Pg. 6 ¶ 0060 - 0063, Pg. 7 ¶ 0070, Pg. 8 ¶ 0094 - 0096 [“The reference library could be obtained from open sources, licensed from institutions such as universities or hospital systems, or a combination thereof… The reference library, in whole or in part, could be generated from open sources such as The Cancer Genome Atlas, California Tumor Tissue Registry, etc., or from particular medical institutions 320 and 322 such as hospitals or universities, the U.S. Government or department thereof, such as the Centers For Disease Control, National Institutes of Health, the U.S. Navy, or combinations thereof” and “The user is given a choice of databases to search in, by checking the box 910, 912 next the name of the institution or source of the images in the reference library”]) and providing, by the computing system, the plurality of different visual search results for display with the skin condition information. (Peng et al., Figs. 1, 2, 10 & 17, Pg. 1 ¶ 0003 and 0006 - 0012, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0050, Pg. 6 ¶ 0063 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0089 - 0092 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102)
- With regards to claim 13, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the method of claim 12, wherein the plurality of condition datasets were at least one of generated or reviewed by a licensed dermatologist. (Peng et al., Pg. 3 ¶ 0040 - 0043 and 0046 - 0047, Pg. 4 ¶ 0051, Pg. 5 ¶ 0054 and 0059, Pg. 6 ¶ 0063, Pg. 7 ¶ 0070, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102 [“generation of semantic similarity labels for medical images can be curated manually or generated from crowdsourced training based on a pathologist click behavior when viewing collections of medical images or other machine learning technique”, “reference library, in whole or in part, could be generated from open sources such as The Cancer Genome Atlas, California Tumor Tissue Registry, etc., or from particular medical institutions 320 and 322 such as hospitals or universities, the U.S. Government or department thereof, such as the Centers For Disease Control, National Institutes of Health, the U.S. Navy, or combinations thereof”, “Dr. Lee is a general dermatologist who has a patient with a suspicious mole. She has arrived at a diagnosis, but wants to double check her assessment… 2. A pivot table (e.g., FIG. 1) is presented of the e.g., 50 cases that were identified as morphologically similar. The table shows the diagnosis & percent of cases with that diagnosis. 3. If most of the identified cases are in line with her assessment, she's reassured that she made the right diagnosis. If most of historical cases fall into another diagnostic category, she can take another look at her current case and compare it to historical cases to make a final diagnosis… 4. She also wants to be able to browse the cases identified to compare them manually to the current case. Hence, she selects any of the cases in the pivot table and browses the images and reviews associated metadata.”])
- With regards to claim 14, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the method of claim 12, wherein providing, by the computing system, the plurality of different visual search results for display with the skin condition information comprises: ordering, by the computing system and via a ranking engine, the plurality of different visual search results based on: a determined visual similarity with the one or more images; (Peng et al., Abstract, Figs. 6, 7 & 10, Pg. 1 ¶ 0003 and 0007 - 0010, Pg. 5 ¶ 0054 - 0055 and 0059, Pg. 6 ¶ 0061 and 0063, Pg. 8 ¶ 0094 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102, Pg. 9 ¶ 0110) and a determined topic relevance based on whether a particular visual search result is associated with the one or more candidate skin conditions. (Peng et al., Abstract, Figs. 6, 7 & 10, Pg. 1 ¶ 0003 and 0008 - 0012, Pg. 5 ¶ 0054 - 0055 and 0059, Pg. 6 ¶ 0061 and 0063, Pg. 7 ¶ 0067 - 0070 and 0081, Pg. 8 ¶ 0094 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102, Pg. 9 ¶ 0110)
- With regards to claim 15, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the method of claim 11, wherein processing, by the computing system, the one or more images with the dermatology conditions classification model to generate the one or more predicted condition classifications comprises: processing, by the computing system, the one or more images with the dermatology conditions classification model to generate a plurality of predicted condition classifications, (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0008 and 0012, Pg. 2 ¶ 0016 - 0020, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050, Pg. 5 ¶ 0054, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0070, Pg. 8 ¶ 0088 - 0091 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100) wherein each predicted condition classification of the plurality of predicted condition classifications is descriptive of a particular candidate skin condition; (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0004, 0008 and 0012, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050 - 0051, Pg. 6 ¶ 0062 - 0065, Pg. 8 ¶ 0088 - 0091 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102) obtaining, by the computing system, a plurality of skin information datasets associated with the plurality of predicted condition classifications, (Peng et al., Figs. 3, 8 & 9, Pg. 1 ¶ 0008 and 0012 - 0013, Pg. 3 ¶ 0040 - 0044 and 0046 - 0047, Pg. 4 ¶ 0049 - 0051, Pg. 6 ¶ 0060 and 0062 - 0065, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102) wherein each skin information dataset of the plurality of skin information datasets is associated with a different candidate skin condition; (Peng et al., Figs. 1, 3 - 5 & 10, Pg. 1 ¶ 0008 and 0011 - 0013, Pg. 3 ¶ 0040 - 0044 and 0046 - 0047, Pg. 4 ¶ 0051 - Pg. 5 ¶ 0055, Pg. 5 ¶ 0059 - Pg. 6 ¶ 0060, Pg. 6 ¶ 0062 - Pg. 7 ¶ 0070, Pg. 8 ¶ 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102) and providing, by the computing system, the plurality of skin information datasets for display. (Peng et al., Figs. 1, 2, 10 & 17, Pg. 1 ¶ 0003 and 0006 - 0012, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0050, Pg. 6 ¶ 0063 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0089 - 0092 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102) Peng et al. fail to disclose explicitly displaying via a carousel interface. However, the Examiner takes official notice of the fact that displaying search results, such as datasets or images, via a carousel interface is notoriously well-known in the art. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Peng et al. in view of Lyman et al. in view of Regan et al. to include a carousel interface for displaying the obtained datasets. This modification would have been prompted in order to substitute the retrieval panes or pivot tables of Peng et al. for a notoriously well-known carousel interface. A notoriously well-known carousel interface could be substituted in place of the retrieval panes or pivot tables of Peng et al. using well-known techniques in the art and would likely yield predictable results, in that, in the combination, the plurality of skin information datasets would be displayed via a carousel interface. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the combined base device would provide the plurality of skin information datasets for display via a carousel interface. Therefore, it would have been obvious to combine Peng et al. in view of Lyman et al. in view of Regan et al. with the notoriously well-known technique of displaying datasets via a carousel interface to obtain the invention as specified in claim 15. The Examiner asserts that the common knowledge or well-known in the art statement is taken to be admitted prior art because applicant failed to traverse the examiner’s assertion of official notice.
- With regards to claim 16, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the method of claim 11, wherein obtaining, by the computing system, the search query comprises: obtaining, by the computing system, the search query via a user interface of a visual search application; (Peng et al., Abstract, Figs. 1, 2, 6 - 10 & 17, Pg. 1 ¶ 0003 and 0006, Pg. 3 ¶ 0040 - 0043, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 - 0063, Pg. 7 ¶ 0082 - 0083, Pg. 8 ¶ 0092 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0104, Pg. 9 ¶ 0109 - 0110) and wherein providing, by the computing system, the skin condition information associated with the one or more candidate skin conditions comprises: providing, by the computing system, the skin condition information for display via the user interface of the visual search application. (Peng et al., Figs. 1, 2, 10 & 17, Pg. 1 ¶ 0003 and 0006 - 0012, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0050, Pg. 6 ¶ 0063 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0089 - 0092 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102) In addition, analogous art Lyman et al. disclose wherein obtaining, by the computing system, the search query comprises: obtaining, by the computing system, the search query via a user interface of a visual search application; (Lyman et al., Figs. 15A - 15C, Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 5 ¶ 0054 - 0055, Pg. 46 ¶ 0323 - 0324, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0339 - 0342, Pg. 57 ¶ 0392 - Pg. 58 ¶ 0398) and wherein providing, by the computing system, the skin condition information associated with the one or more candidate skin conditions comprises: providing, by the computing system, the skin condition information for display via the user interface of the visual search application. (Lyman et al., Figs. 15A - 16A & 16D, Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 5 ¶ 0054 - 0055, Pg. 8 ¶ 0070 - 0075, Pg. 46 ¶ 0323 - 0326, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342, Pg. 51 ¶ 0350 - 0353, Pg. 57 ¶ 0392 - Pg. 58 ¶ 0398)
- With regards to claim 17, Peng et al. disclose instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, (Peng et al., Abstract, Fig. 8, Pg. 1 ¶ 0003, Pg. 2 ¶ 0016, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 and 0065, Pg. 8 ¶ 0088, Pg. 9 ¶ 0105 [“computer system”]) the operations comprising: obtaining a search query (Peng et al., Abstract, Figs. 1, 2, 9 - 12 & 17, Pg. 1 ¶ 0003 - 0004, Pg. 3 ¶ 0040 - 0042, Pg. 4 ¶ 0049 - 0051, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0071, Pg. 8 ¶ 0092 - 0099, Pg. 9 ¶ 0102 - 0105) from a user computing system, (Peng et al., Abstract, Fig. 8, Pg. 1 ¶ 0003, Pg. 2 ¶ 0016, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 and 0065, Pg. 8 ¶ 0088, Pg. 9 ¶ 0105 [“computer system”]) wherein the search query comprises one or more images, wherein the one or more images depict one or more body parts of a user; (Peng et al., Abstract, Figs. 1, 2, 9 - 12 & 17, Pg. 1 ¶ 0003 - 0004, Pg. 3 ¶ 0040 - 0042, Pg. 4 ¶ 0049 - 0051, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0071, Pg. 8 ¶ 0092 - 0099, Pg. 9 ¶ 0102 - 0105) wherein the search query has a diagnostic search intent; (Peng et al., Figs. 1, 2, 10, 11, 13 & 14, Pg. 1 ¶ 0008 and 0011 - 0012, Pg. 2 ¶ 0017 - 0020, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050 - 0051, Pg. 5 ¶ 0054, Pg. 6 ¶ 0062 - Pg. 7 ¶ 0068, Pg. 7 ¶ 0070, Pg. 8 ¶ 0089 - 0091, 0093 and 0096 - 0099, Pg. 9 ¶ 0102 and 0110) providing the one or more images to a medical conditions classification model based on the diagnostic search intent; (Peng et al., Abstract, Figs. 6 - 11, Pg. 1 ¶ 0003 - 0005, Pg. 2 ¶ 0016 - 0020, Pg. 3 ¶ 0045 - 0047, Pg. 4 ¶ 0049 - 0050, Pg. 5 ¶ 0054 - 0059, Pg. 6 ¶ 0061 - 0063 and 0065, Pg. 7 ¶ 0083 - Pg. 8 ¶ 0086, Pg. 8 ¶ 0088) processing the one or more images with the medical conditions classification model to generate one or more predicted condition classifications, (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0008 and 0012, Pg. 2 ¶ 0016 - 0020, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050, Pg. 5 ¶ 0054, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0070, Pg. 8 ¶ 0088 - 0091 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100) wherein the one or more predicted condition classifications are descriptive of one or more candidate medical conditions determined to be potentially depicted in the one or more images; (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0008 and 0012, Pg. 3 ¶ 0040 - 0044 and 0047, Pg. 4 ¶ 0050, Pg. 6 ¶ 0063 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0088 - 0091 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100) obtaining condition information for the one or more candidate medical conditions, (Peng et al., Figs. 3, 8 & 9, Pg. 1 ¶ 0012 - 0013, Pg. 3 ¶ 0046 - 0047, Pg. 4 ¶ 0051, Pg. 6 ¶ 0062 - 0065) wherein the condition information comprises one or more example images of a particular candidate medical condition and a condition name; (Peng et al., Figs. 1, 2 & 10 - 12, Pg. 1 ¶ 0006 - 0008 and 0011 - 0013, Pg. 3 ¶ 0040 - 0044 and 0046 - 0047, Pg. 4 ¶ 0050 - 0051, Pg. 6 ¶ 0062 - 0065, Pg. 7 ¶ 0069 - 0070, Pg. 8 ¶ 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102) processing the one or more images with a search engine to determine one or more visual search results, (Peng et al., Abstract, Figs. 1 - 10 & 17, Pg. 1 ¶ 0003 - 0008 and 0013, Pg. 3 ¶ 0040 - 0044 and 0046, Pg. 4 ¶ 0050, Pg. 5 ¶ 0052 - 0055 and 0059, Pg. 6 ¶ 0061 - 0062, Pg. 8 ¶ 0084 - 0088 and 0094 - 0096) wherein the one or more visual search results are determined based on a visual feature similarity with the one or more images; (Peng et al., Abstract, Figs. 6, 7 & 10, Pg. 1 ¶ 0003 and 0007 - 0010, Pg. 5 ¶ 0054 - 0055 and 0059, Pg. 6 ¶ 0061 and 0063, Pg. 8 ¶ 0094 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102, Pg. 9 ¶ 0110) and providing the one or more visual search results and the condition information to the user computing system. (Peng et al., Figs. 1, 2, 10 & 17, Pg. 1 ¶ 0003 and 0006 - 0012, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0050, Pg. 6 ¶ 0063 - 0065, Pg. 7 ¶ 0070, Pg. 8 ¶ 0089 - 0092 and 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0102) Peng et al. fail to disclose explicitly one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations; processing the one or more images with an intent classification model to generate an intent classification, wherein the intent classification indicates that the search query has a diagnostic search intent, wherein the intent classification model was trained to determine a search intent of the user based on one or more features in an input image; determining to perform a dedicated diagnostic processing flow instead of general search engine processing based on the diagnostic search intent, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet; and in response to determining a dedicated diagnostic processing flow based on the diagnostic search intent, providing the images to the medical condition classification model based on the intent classification. Pertaining to analogous art, Lyman et al. disclose one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, (Lyman et al., Pg. 1 ¶ 0034 - Pg. 2 ¶ 0037, Pg. 4 ¶ 0050, Pg. 57 ¶ 0392, Pg. 57 ¶ 0397 - Pg. 58 ¶ 0398) the operations comprising: obtaining a search query from a user computing system, wherein the search query comprises one or more images, wherein the one or more images depict one or more body parts of a user; (Lyman et al., Figs. 1 - 3 & 15A - 16A, Pg. 14 ¶ 0108 - 0110, Pg. 46 ¶ 0323 - Pg. 47 ¶ 0332, Pg. 48 ¶ 0334 - 0336, Pg. 49 ¶ 0340 - 0342) processing the one or more images with an intent classification model to generate an intent classification, (Lyman et al., Pg. 46 ¶ 0324 - 0325, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“Each processing function 4008 can utilize a computer vision model trained to diagnose conditions of the corresponding type of image, for example, where each processing function 4008 is trained on a training set of images of the corresponding type as input data and on corresponding diagnosis data as output labels” and “multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) wherein the intent classification indicates that the search query has a diagnostic search intent, (Lyman et al., Pg. 46 ¶ 0324 - 0325, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“Each processing function 4008 can utilize a computer vision model trained to diagnose conditions of the corresponding type of image, for example, where each processing function 4008 is trained on a training set of images of the corresponding type as input data and on corresponding diagnosis data as output labels” and “multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) wherein the intent classification model was trained to determine a search intent of the user based on one or more features in an input image; (Lyman et al., Pg. 46 ¶ 0324 - 0325, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“Each processing function 4008 can utilize a computer vision model trained to diagnose conditions of the corresponding type of image, for example, where each processing function 4008 is trained on a training set of images of the corresponding type as input data and on corresponding diagnosis data as output labels”]) determining to perform a dedicated diagnostic processing flow based on the diagnostic search intent; (Lyman et al., Pg. 46 ¶ 0324 - Pg. 47 ¶ 0331, Pg. 49 ¶ 0341 - 0342 [“the captured image processing system 4004 first performs a image classifier function 4007, for example, to determine an anatomical region of the image, to determine anatomical features in the image that need to be diagnosed, and/or to otherwise classify the image into one of a plurality of image types. Once the image type is determined by performing the image classifier function 4007 on the captured image, one of a plurality of different processing functions 4008, each corresponding to one of the plurality of image types, can be selected for performance on the image data”, “image classifier function 4007 can detect the anatomical region, type of bodily tissue and/or bodily fluid captured, and/or other classifying information for the captured image to determine the processing function 4008” and “multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) in response to determining a dedicated diagnostic processing flow based on the diagnostic search intent, providing the one or more images to a medical conditions classification model based on the intent classification; (Lyman et al., Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342 [“multiple processing functions 4008 are performed in accordance with a hierarchy of processing functions 4008. For example, a first processing function 4008 trained to detect skin conditions is performed first on the captured image in response to determining the captured image is a picture of the skin. The diagnosis data generated by the first processing function can indicates a rash, and second processing functions 4008 trained to diagnose particular types of rashes can be selected to be performed next on the captured image to provide a more detailed diagnosis”]) and processing the one or more images with the medical conditions classification model to generate one or more predicted condition classifications, (Lyman et al., Figs. 15A - 15C, Pg. 8 ¶ 0070 - 0075, Pg. 46 ¶ 0323 - 0326, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342, Pg. 51 ¶ 0350 - 0353) wherein the one or more predicted condition classifications are descriptive of one or more candidate medical conditions determined to be potentially depicted in the one or more images. (Lyman et al., Figs. 15A - 15C, Pg. 8 ¶ 0070 - 0075, Pg. 46 ¶ 0323 - 0326, Pg. 47 ¶ 0328 - 0331, Pg. 49 ¶ 0341 - 0342, Pg. 51 ¶ 0350 - 0353) Lyman et al. fail to disclose explicitly performing the dedicated processing flow instead of general search engine processing, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet. Pertaining to analogous art, Regan et al. disclose obtaining a search query; (Regan et al., Abstract, Figs. 2, 3, 8 & 12, Pg. 1 ¶ 0011 - 0013, Pg. 3 ¶ 0046, Pg. 4 ¶ 0054 - Pg. 5 ¶ 0055, Pg. 5 ¶ 0059 - 0061) processing the search query to generate an intent classification, (Regan et al., Figs. 2 & 12, Pg. 3 ¶ 0046, Pg. 4 ¶ 0049, Pg. 4 ¶ 0054 - Pg. 5 ¶ 0055, Pg. 9 ¶ 0096 - 0101) wherein the intent classification indicates that the search query has a search intent; (Regan et al., Figs. 2 & 12, Pg. 3 ¶ 0046, Pg. 4 ¶ 0049, Pg. 4 ¶ 0054 - Pg. 5 ¶ 0055, Pg. 9 ¶ 0096 - 0101, Pg. 10 ¶ 0105 - 0108, Pg. 19 ¶ 0111 - Pg. 20 ¶ 0114) and determining to perform a dedicated diagnostic processing flow instead of general search engine processing based on the search intent, (Regan et al., Figs. 2, 8 & 12, Pg. 1 ¶ 0009 - 0013, Pg. 3 ¶ 0046, Pg. 4 ¶ 0049 - 0050, Pg. 10 ¶ 0105 - Pg. 19 ¶ 0111, Pg. 20 ¶ 0114 - 0115) wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet. (Regan et al., Figs. 2, 3, 8 & 9, Pg. 1 ¶ 0003 - 0009, Pg. 20 ¶ 0115) Peng et al. and Lyman et al. are combinable because they are both directed towards medical image processing systems that process dermatology images depicting skin to generate predicted diagnosis data for the dermatology images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Peng et al. with the teachings of Lyman et al. A first modification would have been prompted in order to enhance the base device of Peng et al. with the well-known and applicable technique Lyman et al. applied to a comparable device. Utilizing one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, as taught by Lyman et al., to implement operations of the base device of Peng et al. would enhance the base device of Peng et al. by ensuring that its operations are carried out accurately and efficiently at high-computational speed on computer architecture, by facilitating widespread distribution of the base device of Peng et al. to millions of potential end-users with access to a computer and by simplifying the process of making revisions, modifications and/or updates to the operations of the base device of Peng et al. A second modification would have been prompted in order to enhance the base device of Peng et al. with the well-known and applicable technique Lyman et al. applied to a comparable device. Processing the one or more images with an intent classification model to generate an intent classification and providing the one or more images to the medical conditions classification model based on the intent classification, as taught by Lyman et al., would enhance the base device of Peng et al. by reducing the total number of image processing operations it performs since only search query images determined to have a diagnostic search intent would undergo processing by the medical conditions classification model thereby eliminating unnecessary image processing operations from being performed by the medical conditions classification model on search query images lacking diagnostic search intent and improving the overall computational efficiency of the base device of Peng et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations would be utilized to implement the base device of Peng et al. so as to ensure that its operations are carried out accurately and efficiently at high-computational speed on computer architecture, to facilitate widespread distribution of the base device of Peng et al. to millions of potential end-users with access to a computer and to simplify the process of making revisions, modifications and/or updates to the operations of the base device of Peng et al. and in that the one or more images would be processed with an intent classification model to generate an intent classification and provided to the medical conditions classification model based on the intent classification so as to help ensure that unnecessary image processing operations are not performed by the medical conditions classification model on search query images lacking a diagnostic search intent and thereby improve the overall computational efficiency of the base device of Peng et al. In addition, Peng et al. in view of Lyman et al. and Regan et al. are combinable because they are all directed towards systems and methods that process search queries of users to return the most pertinent and relevant search results possible. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Peng et al. in view of Lyman et al. with the teachings of Regan et al. This modification would have been prompted in order to enhance the combined base device of Peng et al. in view of Lyman et al. with the well-known and applicable technique of Regan et al. Performing the dedicated processing flow instead of general search engine processing based on the search intent, wherein general search engine processing comprises determining one or more general search results obtained from web resources across an internet, as taught by Regan et al., would enhance the combined base device by helping ensure that users are quickly and efficiently provided with the most relevant information possible in response to their search queries since in cases where the intent of their search queries cannot be definitively determined general search engine processing would instead be performed and/or by allowing for users to perform searches for an increased number and/or variety of different types of information thereby improving the overall appeal, usefulness and marketability of the combined base device to potential end-users. Furthermore, this modification would have been prompted by the teachings and suggestions of Peng et al. that tools for searching for medical images include commercially available products or services such as Google’s Image Search function, that the reference library they search can be generated from a variety of open sources, that the reference library can be contained on remote server, that a user is given a choice of databases to search, and that search results can be filtered by image source and/or various metadata associated with the images, see at least figures 3 and 8 - 10, page 1 paragraph 0002, page 4 paragraph 0051 and page 6 paragraphs 0061 - 0063 of Peng et al. Moreover, this modification would have been prompted by the teachings and suggestions of Lyman et al. that captured image and/or diagnosis data can alternatively or additionally be transmitted to any other medical entity such as a doctor of the patient and/or to another client device to generate additional diagnosis data, and that in some embodiments a single processing function 4008 is performed on all captured images and/or no image classifier function 4007 is performed, see at least page 46 paragraph 0326 and page 47 paragraph 0331 of Lyman et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that in situations wherein a search query is not indicated as having a diagnostic search intent the combined base device would perform general search engine processing instead of the dedicated diagnostic processing flow so as to help ensure that users are quickly and efficiently provided with the most relevant and pertinent information possible in response to their search queries. Therefore, it would have been obvious to combine Peng et al. with Lyman et al. and Regan et al. to obtain the invention as specified in claim 17.
- With regards to claim 18, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the one or more non-transitory computer-readable media of claim 17, wherein processing the one or more images with a search engine to determine the one or more visual search results comprises: determining a plurality of candidate visual search results based on the one or more images; (Peng et al., Abstract, Figs. 1 - 10 & 17, Pg. 1 ¶ 0003 - 0008 and 0013, Pg. 3 ¶ 0040 - 0044 and 0046, Pg. 4 ¶ 0050, Pg. 5 ¶ 0052 - 0055 and 0059, Pg. 6 ¶ 0061 - 0062, Pg. 8 ¶ 0084 - 0088 and 0094 - 0096) determining one or more anatomy visual search results of the plurality of candidate visual search results, (Peng et al., Figs. 3 & 13 - 15, Pg. 3 ¶ 0046 - 0047, Pg. 4 ¶ 0050 - 0051, Pg. 5 ¶ 0059 - Pg. 6 ¶ 0060, Pg. 6 ¶ 0063 - 0065, Pg. 7 ¶ 0067 - 0070, Pg. 8 ¶ 0093 - 0096 and 0099, Pg. 9 ¶ 0102 and 0110) wherein the one or more anatomy visual search results comprise one or more anatomy images that depict a human body part; (Peng et al., Figs. 3 & 13 - 15, Pg. 3 ¶ 0046 - 0047, Pg. 4 ¶ 0050 - 0051, Pg. 5 ¶ 0059 - Pg. 6 ¶ 0060, Pg. 6 ¶ 0063 - 0065, Pg. 7 ¶ 0067 - 0070, Pg. 8 ¶ 0093 - 0096 and 0099, Pg. 9 ¶ 0102 and 0110) and wherein the one or more visual search results comprise the one or more anatomy visual search results. (Peng et al., Figs. 1 - 3, 10 - 12 & 17, Pg. 1 ¶ 0006 - 0012, Pg. 3 ¶ 0043 - 0047, Pg. 4 ¶ 0049 - 0051, Pg. 5 ¶ 0054, Pg. 6 ¶ 0061 - 0065, Pg. 7 ¶ 0067 - 0070, Pg. 8 ¶ 0094 - 0096, Pg. 9 ¶ 0101 - 0104 and 0110)
- With regards to claim 19, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the one or more non-transitory computer-readable media of claim 17, wherein processing the one or more images with a search engine to determine one or more visual search results comprises: determining a plurality of candidate visual search results based on the one or more images; (Peng et al., Abstract, Figs. 1 - 10 & 17, Pg. 1 ¶ 0003 - 0008 and 0013, Pg. 3 ¶ 0040 - 0044 and 0046, Pg. 4 ¶ 0050, Pg. 5 ¶ 0052 - 0055 and 0059, Pg. 6 ¶ 0061 - 0062, Pg. 8 ¶ 0084 - 0088 and 0094 - 0096) determining one or more particular candidate search results of the plurality of candidate visual search results are associated with the one or more candidate medical conditions; (Peng et al., Abstract, Figs. 1, 2, 6, 7, 10 & 11, Pg. 1 ¶ 0003 - 0012, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0050, Pg. 5 ¶ 0052 - 0055 and 0059, Pg. 6 ¶ 0061 - 0065, Pg. 7 ¶ 0067 - 0070, Pg. 8 ¶ 0084 - 0087 and 0094 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0104) and adjusting a ranking of the plurality of candidate visual search results based on determining the one or more particular candidate search results are associated with the one or more candidate medical conditions. (Peng et al., Abstract, Figs. 6, 7 & 10, Pg. 1 ¶ 0003 and 0005 - 0012, Pg. 5 ¶ 0054 - 0055, Pg. 5 ¶ 0059 - Pg. 6 ¶ 0061, Pg. 6 ¶ 0063, Pg. 7 ¶ 0068 - 0070 and 0081, Pg. 8 ¶ 0096, Pg. 9 ¶ 0100 - 0104 [“The computer system is trained to find one or more similar medical images in the reference library system which are similar to the input image. The reference library is represented as an embedding of each of the medical images projected in a feature space having a plurality of axes, wherein the embedding is characterized by two aspects of a similarity ranking: (1) visual similarity, and (2) semantic similarity such that neighboring images in the feature space are visually similar and semantic information is represented by the axes of the feature space”])
- With regards to claim 20, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the one or more non-transitory computer-readable media of claim 17, wherein the one or more visual search results are provided in a first panel of a search results interface, (Peng et al., Figs. 1, 2, 10, 11 & 17, Pg. 1 ¶ 0006 - 0008 and 0011, Pg. 2 ¶ 0022 - 0024 and 0032 - 0033, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0061 - 0065, Pg. 8 ¶ 0086 - 0087 and 0094 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100, Pg. 9 ¶ 0102 and 0110) and wherein the condition information is provided in a second panel of the search results interface. (Peng et al., Figs. 1, 2, 10 & 11, Pg. 3 ¶ 0043 - 0044, Pg. 4 ¶ 0049 - 0050, Pg. 6 ¶ 0063 - 0065, Pg. 8 ¶ 0094 - 0096, Pg. 8 ¶ 0099 - Pg. 9 ¶ 0100)
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Peng et al. U.S. Publication No. 2021/0019342 A1 in view of Lyman et al. U.S. Publication No. 2020/0160980 A1 in view of Regan et al. U.S. Publication No. 2013/0019202 A1 as applied to claim 9 above, and further in view of Kondo et al. U.S. Publication No. 2017/0091413 A1.
- With regards to claim 10, Peng et al. in view of Lyman et al. in view of Regan et al. disclose the system of claim 9, wherein the operations further comprise: obtaining a selection input associated with the selectable user interface element, wherein the selection input is descriptive of a selection of the selectable user interface element; (Peng et al., Figs. 1, 10 & 17, Pg. 1 ¶ 0008 - 0012, Pg. 4 ¶ 0048 - 0049, Pg. 6 ¶ 0061 - 0064, Pg. 8 ¶ 0086 - 0087, 0094 - 0096 and 0099) processing the particular medical condition with a search engine to determine a plurality of updated search results associated with the particular medical condition; (Peng et al., Figs. 1, 2, 10 & 11, Pg. 1 ¶ 0009 - 0012, Pg. 4 ¶ 0048, Pg. 6 ¶ 0063 - 0064, Pg. 8 ¶ 0086 - 0087, 0094 - 0096 and 0099, Pg. 9 ¶ 0110 [“task bar 1004 provides tools for filtering the retrieved images, e.g., on the basis of metadata associated with each of the images. For example, the user could select filters such that only images of cancerous lesions are displayed, only images from male or female patients, only images with a survival time of less than 5 years, etc. In the results region 1010, the user is given an option to show statistics of the returned images, such as the top diagnoses for all 50 images returned, which are listed in the region 1010. A drop-down box 1012 allow for the user to select other statistics or summaries of the returned images. The menu of options available in the drop-down box will vary on the type of images and persons skilled in the art will be able to arrive at a suitable menu of options depending on implementation details which are not particularly important. Examples of such options include Diagnosis” and “tools on the display could interactively adjust the similarity metric based on the user's input, to facilitate an iterative search. In particular, the result set can be refined based on the previous choices of the user. If, for example, the user selected an image of similar cancer grade (but dissimilar histological features), more images of the similar cancer grade could be shown by emphasizing that particular dimension in the metric embedding space for the neighbor list search”]) and providing the plurality of updated search results for display. (Peng et al., Figs. 1, 2, 10, 11 & 17, Pg. 1 ¶ 0009 - 00012, Pg. 2 ¶ 0022, Pg. 4 ¶ 0048 - 0050, Pg. 6 ¶ 0061 - 0064, Pg. 8 ¶ 0086 - 0087, 0094 - 0096 and 0099, Pg. 9 ¶ 0110) Peng et al. fail to disclose explicitly processing a condition name of the particular medical condition to determine the plurality of updated search results. Pertaining to analogous art, Kondo et al. disclose processing a condition name of the particular medical condition with a search engine to determine a plurality of updated search results associated with the particular medical condition; (Kondo et al., Figs. 45 & 46, Pg. 2 ¶ 0053 - 0054, Pg. 5 ¶ 0125 - 0128, Pg. 10 ¶ 0196 - Pg. 11 ¶ 0203, Pg. 18 ¶ 0300 - Pg. 19 ¶ 0308, Pg. 20 ¶ 0326 - Pg. 21 ¶ 0329, Pg. 21 ¶ 0331, Pg. 24 ¶ 0381 - Pg. 25 ¶ 0389) and providing the plurality of updated search results for display. (Kondo et al., Abstract, Figs. 45 & 46, Pg. 2 ¶ 0053 - 0054, Pg. 5 ¶ 0128, Pg. 10 ¶ 0196 - Pg. 11 ¶ 0205, Pg. 19 ¶ 0307 - 0309, Pg. 20 ¶ 0327 - Pg. 21 ¶ 0331, Pg. 24 ¶ 0381 - Pg. 25 ¶ 0389) Peng et al. in view of Lyman et al. in view of Regan et al. and Kondo et al. are combinable because they are all directed towards systems and methods that process search queries of users to return the most pertinent and relevant search results possible and, similar to Peng et al. and Lyman et al., Kondo et al. is also directed towards a medical image processing system that processes search query images to generate predicted diagnosis data for the search query images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Peng et al. in view of Lyman et al. in view of Regan et al. with the teachings of Kondo et al. This modification would have been prompted in order to substitute the search results filtering/refining technique of Peng et al. for the search results refining/updating process of Kondo et al. The search results refining/updating process of Kondo et al. could be substituted in place of the search results filtering/refining technique of Peng et al. using well-known techniques in the art and would likely yield predictable results, in that, in the combination, the search results refining / updating process of Kondo et al., which processes a condition name with a search engine, would be utilized to determine the plurality of updated search results associated with the particular medical condition. Furthermore, this modification would have been prompted by the teachings and suggestions of Peng et al. that their user interface may provide tools for filtering and/or refining searches and that the tools may filter retrieved images on the basis of metadata associated with the retrieved images so that, for example, only images of a certain diagnosis are displayed, see at least page 2 paragraph 0022, page 4 paragraphs 0048 - 0049, page 6 paragraph 0063 and page 8 paragraphs 0087 and 0094 - 0096 of Peng et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the combined base device would process a condition name of the particular medical condition with a search engine to determine the plurality of updated search results. Therefore, it would have been obvious to combine Peng et al. in view of Lyman et al. in view of Regan et al. with Kondo et al. to obtain the invention as specified in claim 10.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC RUSH whose telephone number is (571) 270-3017. The examiner can normally be reached 9am - 5pm Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270 - 5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ERIC RUSH/Primary Examiner, Art Unit 2677