DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 9/26/25 has been entered.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 2, 3, 8, 10-13, 25, 27, 29, 30, 34- 36, 39, 42-48, and 51-55 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
The following analysis is based on the 2019 Revised Patent Subject Matter Eligibility Guidance (2019 PEG) published on January 7, 2019 (84 Fed. Reg. 50). See also MPEP 2106.04(a)(2)(II).
Regarding claim 2, 13 and 51:
STEP 1:
Claim 2, 13 and 51 meet step 1 requirement as claims 2 and 13 are directed towards method claims i.e., a process and claim 51 is directed to CRM, i.e., manufacture as statutory subject matter category.
STEP 2A, PRONG 1 test:
Is the claim directed to a law of nature, a natural phenomenon or an abstract idea?
YES, the claims are directed toward a mental process (i.e., abstract idea).
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
• Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
• Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
• Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
The methods in claims 2, 13 and 51 comprise mental processes that can be practicably performed in the human mind therefore, an abstract idea as follows:
Claim 2 (and 51):
executing,mental process as an abstract idea (concepts performed in the human mind including an observation, evaluation, judgment, opinion see MPEP § 2106.04(a)(2), subsection III).
generating a modified captured image by modifying, based on the one or more instructions, the at least one portion of the recognized object in the captured image (a human can readily draw a modified image (e.g., an edited drawing of a hat, a shirt, etc.), using a pen and paper, as a mental process as an abstract idea (concepts performed in the human mind including an observation, evaluation, judgment, opinion see MPEP § 2106.04(a)(2), subsection III).
Claim 13:
executingmental process as an abstract idea (concepts performed in the human mind including an observation, evaluation, judgment, opinion see MPEP § 2106.04(a)(2), subsection III).
generating a modified captured image by replacing, based on the instructions, the one or more portions of the recognized pattern in the captured image with the replacement image (a human can readily draw a modified image (e.g., an edited drawing of a hat, a shirt, etc.) and patterns thereof, using a pen and paper, as a mental process as an abstract idea (concepts performed in the human mind including an observation, evaluation, judgment, opinion see MPEP § 2106.04(a)(2), subsection III).
These limitations, as drafted, is a simple process that, under their broadest reasonable interpretation, covers performance of the limitations in the mind or by a human. The Examiner notes that under MPEP 2106.04(a)(2)(III), the courts consider a mental process (thinking) that “can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same).
The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide rule) to perform the claim limitation. See, e.g., Benson, 409 U.S. at 67, 65, 175 USPQ at 674-75, 674 (noting that the claimed "conversion of [binary-coded decimal] numerals to pure binary numerals can be done mentally," i.e., "as a person would do it by head and hand."); Synopsys, Inc. v. Mentor Graphics Corp., 839 F.3d 1138, 1139, 120 USPQ2d 1473, 1474 (Fed. Cir. 2016) (holding that claims to a mental process of "translating a functional description of a logic circuit into a hardware component description of the logic circuit" are directed to an abstract idea, because the claims "read on an individual performing the claimed steps mentally or with pencil and paper").
Nor do the courts distinguish between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer. As the Federal Circuit has explained, "[c]ourts have examined claims that required the use of a computer and still found that the underlying, patent-ineligible invention could be performed via pen and paper or in a person’s mind." Versata Dev. Group v. SAP Am., Inc., 793 F.3d 1306, 1335, 115 USPQ2d 1681, 1702 (Fed. Cir. 2015). See also Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d 1307, 1318, 120 USPQ2d 1353, 1360 (Fed. Cir. 2016) (‘‘[W]ith the exception of generic computer-implemented steps, there is nothing in the claims themselves that foreclose them from being performed by a human, mentally or with pen and paper.’’); Mortgage Grader, Inc. v. First Choice Loan Servs. Inc., 811 F.3d 1314, 1324, 117 USPQ2d 1693, 1699 (Fed. Cir. 2016) (holding that computer-implemented method for "anonymous loan shopping" was an abstract idea because it could be "performed by humans without a computer").
Because both product and process claims may recite a "mental process", the phrase "mental processes" should be understood as referring to the type of abstract idea, and not to the statutory category of the claim. The courts have identified numerous product claims as reciting mental process-type abstract ideas, for instance the product claims to computer systems and computer-readable media in Versata Dev. Group. v. SAP Am., Inc., 793 F.3d 1306, 115 USPQ2d 1681 (Fed. Cir. 2015).
As such, a human can visually recognize an object (e.g., a hat, a shirt, etc.) in an image/ drawing as a mental process as an abstract idea (concepts performed in the human mind including an observation, evaluation, judgment, opinion see MPEP § 2106.04(a)(2), subsection III). The mere nominal recitation that the various steps are being executed by a computing device (e.g., a generic computer) does not take the limitations out of the mental process grouping. Thus, the claims recite a mental process.
If a claim limitation, under its BRI (broadest reasonable interpretation) covers performance of a mental step which could be performed with a simple tool such as a pen/ marker and paper (e.g., hand drawing), then it falls within the “mental steps” grouping of abstract ideas. Accordingly, the claims recite an abstract idea.
Step 2A, prong 2 test:
Does the claim recite additional elements that integrate the judicial exception into a practical application? No; as explained below.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
• an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
• an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
• an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
• an additional element effects a transformation or reduction of a particular article to a different state or thing; and
• an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
• an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
• an additional element adds insignificant extra-solution activity to the judicial exception; and
• an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claims 2, 13 and 51 further recite as follows:
Claim 2 (and 51):
sending, to a remote database storing a plurality of modification preferences for a plurality of objects, data associated with the recognized object, wherein a first modification preference of the plurality of modification preferences comprises a user privacy preference indicating that at least one portion of the recognized object in captured images comprising the recognized object should be blocked from display by one or more displays of the computing device (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea);
receiving, from the remote database, one or more instructions for modifying one or more portions of the recognized object based on the first modification preference (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea);
causing output, by the one or more displays of the computing device and based on the one or more instructions, at the modified captured image (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea).
Claim 13:
sending, to a remote database storing a plurality of modification preferences for a plurality of patterns, data associated with the recognized pattern (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea);
receiving, from the remote database, instructions for modifying one or more portions of the recognized pattern, wherein at least a portion of the instructions are configured to replace the one or more portions of the recognized pattern with a replacement image (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea);
causing output, by one or more displays of the computing device and based on the | instructions, of the modified captured image (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea).
The claims further recite additional elements as – a “computing device”, a “remote database” and “one or more displays of the computing device” to perform various tasks. The additional identified elements supra are "Merely adding a generic computer, generic computer components, or a programmed computer to perform generic computer functions” and does not automatically overcome an eligibility rejection. The “remote database” in the claims is performing generic computer functions, not qualifying as a particular machine.
Paragraphs [0036- 37] of the specification disclose the “computing device” as “The computing device 200 may include one or more processors 201, which may execute instructions of a computer program to perform any of the features described herein”. Accordingly, the “computing device” is merely a generic computer using generic computing devices and components thereof (e.g., the processor 201, the ROM storage 202, the display 206, etc.) as depicted in figure 2.
The “one or more displays of the computing device” are further disclosed and exemplified in paragraphs [0035, and 39] of the specification as generic display devices.
Assuming arguendo even if the specifics were added, the applicant did not invent or improve the “computing devices” and other components thereof (i.e., one or more displays of the computing device). Instead, the “computing device” and components thereof stand in for the automation of an otherwise human mental process. With respect to the function of “display”, the broadest reasonable interpretation (BRI) would have encompassed any forms of displaying inclusive of manually presenting/ displaying. The claim does not preclude manual presentation and/ or display options. Moreover, the display/ GUI (graphic user interface) is merely a generic display and also constitutes a standard computer interface to implement an otherwise abstract idea. See MPEP 2106.04(a)(2)(II).
The “remote database” is further disclosed and exemplified as a generic database/ storage throughout the specification. The remote database is merely performing generic computer functions, not qualifying as a particular machine.
STEP 2, Prong 2B test:
Does the claim recite additional elements that amount to significantly more than the judicial exception?
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
With regard to (2b) the Guidance provided the following examples of limitations that may be enough to qualify as “significantly more" when recited in a claim with a judicial exception:
Improvement to another technology or technical field
Improvement to functioning of computer itself and/or applying the judicial exception with, or by use of, a particular machine
Effecting a transformation or reduction of a particular article to a different state or thing.
Adding a specific limitation other that what is well understood, routine and conventional in the field, or adding unconventional steps that confine the claim to a particular useful application
Meaningful limitation beyond generally linking the use of an abstract idea to a particular technological environment.
The Guidance further set forth limitations that were found not to be enough to qualify as “significantly more” when recited in a claim with a judicial exception include:
Adding words to “apply it” (or an equivalent) with the judicial exception or mere instructions to implement abstract ideas on a computer
Simply appending well-understood, routine and conventional activities previously known to the industry specified at a high level of generality to the judicial exception, e.g., a claim to an abstract idea requiring no more than a generic
Computer to perform generic computer functions that are well -understood, routine and conventional activities previously known to the industry.
Adding insignificant extra-solution activity to the judicial exception, e.g. mere data gathering in conjunction with a law of nature or abstract idea
Generally linking the use of the judicial exception to a particular technological environment or field of use.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The identified additional elements supra are merely a generic computer and components thereof, using instructions to apply the judicial exception, and are well-known, routine, and conventional as is evidenced by Bancorp Services v. Sun Life (Fed. Cir. 2012) and Alice Corp. v. CLS Bank (2014).
Claims 2, 13 and 51 further recite as follows:
Claim 2 (and 51):
sending, to a remote database storing a plurality of modification preferences for a plurality of objects, data associated with the recognized object, wherein a first modification preference of the plurality of modification preferences comprises a user privacy preference indicating that at least one portion of the recognized object in captured images comprising the recognized object should be blocked from display by one or more displays of the computing device (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea);
receiving, from the remote database, one or more instructions for modifying one or more portions of the recognized object based on the first modification preference (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea);
causing output, by the one or more displays of the computing device and based on the one or more instructions, at the modified captured image (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea).
Claim 13:
sending, to a remote database storing a plurality of modification preferences for a plurality of patterns, data associated with the recognized pattern (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea);
receiving, from the remote database, instructions for modifying one or more portions of the recognized pattern, wherein at least a portion of the instructions are configured to replace the one or more portions of the recognized pattern with a replacement image (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea);
causing output, by one or more displays of the computing device and based on the | instructions, of the modified captured image (insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea).
Claims 2, 13 and 51 further recite additional elements as – a “computing device”, a “remote database” and “one or more displays of the computing device” to perform various tasks. The additional identified elements supra are "Merely adding a generic computer, generic computer components, or a programmed computer to perform generic computer functions” and does not automatically overcome an eligibility rejection. The “remote database” in the claims is performing generic computer functions, not qualifying as a particular machine.
The “computing device” further does not qualify as significantly more since the device is merely a generic computer and peripheral components thereof to perform unqualifying generic functions as listed supra. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The identified additional generic computer and components thereof merely provide instructions to apply the judicial exception using generic computer and components thereof that is well-known, routine, and conventional as is evidenced by Bancorp Services v. Sun Life (Fed. Cir. 2012) and Alice Corp. v. CLS Bank (2014).
The display device is further well-known, routine, and conventional as per the following factual evidences:
Paragraph [0008] of the US PAP 2019/ 0004613 discloses “FIG. 1 illustrates a schematic view of a deviation between a position of a gesture detected by a depth camera in a conventional augmented reality display device and a position of the gesture detected by a user eye”.
Paragraph [009] of the US PAP 2021/ 0311311 discloses “An AR display may be head-mounted display (HMD), a Heads-Up Display (HUD) or otherwise a conventional display such as a mobile device or a computer. A conventional AR display architecture uses an additive-only combiner to overlay computer generated virtual content on top of the real-world scene”.
Paragraph [0006] of the US PAP 2022/ 0139279 discloses “However, since a visor has fixed light reflectance, there is a disadvantage in that the visor cannot reflect a change in light transmittance according to changes in external environments such as time and place. Accordingly, the conventional augmented reality display apparatus equipped with a visor, when augmented reality is implemented, a sensation of immersion is decreased”.
Paragraph [0002] of the USPAP 2023/ 0176381 discloses “FIG. 1 is a schematic structural view of a conventional augmented reality display system commonly used for general AR goggles. As shown in FIG. 1, the augmented reality display system 10 includes a curved reflector 12, an optical splitter 14 and an image display 16. An image beam L transmitted from the image display 16 is firstly reflected by the optical splitter 14 to the curved reflector 12 and then is reflected and enlarged by the curved reflector 12. The enlarged image beam L passes through the optical splitter 14 and becomes a to-be-displayed image beam L′ to be received by a human eye 18”.
The remote database is further well-known, routine, and conventional as per the following factual evidences:
Paragraph [0019] pf 2019/ 0163975 discloses “The mobile communication device 100 includes a processor 102, an imaging device 106, and a display 110 along with other conventional components that are part of the mobile communication device 100 such as network interfaces for communicating with remote server systems and databases over a network 120. Other examples of components that can be part of mobile communication device 100 will be described below with reference to example implementations that are shown FIGS. 8 and 9. However, it should be appreciated that the disclosed embodiments can be implemented in conjunction with other types of user systems, such as desktops, laptops, tablets, smartphones or other client devices, Google Glass™, and any other computing device implemented in an automobile, aircraft, television, or other business or consumer electronic device or system, including web clients”.
Paragraph [0047] of 2017/ 0206711 discloses “In one aspect, the storage medium is part of the system. In another aspect, the system transmits the video file, entirely or partially, to a remote server (109). Storage on a remote server can facilitate downloading or playing by another user. In one aspect, the remote server is a conventional database server. In another aspect, the remote server is a cloud server having a distributed system”.
As per dependent claims 3, 8, 10-12, 25, 27, 29, 30, 34- 36, 39, 42-48, and 52-55, the claims do not add anything which would qualify as patent eligible subject matter under 35 USC 101 under the Alice/ Mayo two- part test. The identified additional elements are generic computer and components thereof performing generic computer functions in conjunction with an abstract idea.
As per claims 3 and 52, a human can search and visually identify the recognized object (e.g., a hat, a shirt, etc.) among drawing(s) and can readily decide of a match/ or no match as a mental process. The sending and/ or receiving of data corresponds to an insignificant extra-solution activity to the judicial exception, e.g. mere data gathering in conjunction with a law of nature or abstract idea.
As per claims 8 and 53, a human can visually observe and decide a geographic location based on an image (e.g., look at a photo and see the Washington Nationals stadium in DC) as a mental process.
As per claims 10 and 54, a human can distort an image/ drawing and/ or portions thereof using a marker as a mental process.
As per claims 11 and 55, a human can readily determine modifications in the form of different colors (red, green, blue, etc.) for a plurality of images/ drawings and then modify an output of the object (make the hat red, green and blue with stripes using a marker or by pasting in a new hat that is different colors) using a pen/ marker and paper as a mental process.
As per claim 12, a human can visually check the profile/ shape/ outline of a recognized object (e.g., hat) in an image/ drawing and proceed to make changes/ modifications using a pen/ marker and paper as a mental process.
As per claim 21, a human can distort an image/ drawing and/ or portions thereof inclusive of landmarks within the image (e.g., Eiffel tower, Nationals Stadium in DC, etc.) using a marker as a mental process.
As per claim 25, a human can distort an image/ drawing and/ or portions thereof including a face using a marker as a mental process.
As per claim 27, a human can distort an image/ drawing and/ or portions thereof including patterns using a marker as a mental process.
As per claim 29, a human can edit/ modify/ update an image/ drawing and/ or portions thereof including a face using a marker as a mental process.
As per claim 30, a human can modify an image/ drawing and/ or portions thereof including a scar, birth mark, tattoo, etc. using a marker as a mental process.
As per claim 34, a human can modify an image/ drawing and/ or portions thereof including a face with a painting, tattoo, etc. using a pen/ marker and paper as a mental process.
As per claim 35, a human can modify an image/ drawing and/ or portions thereof to include a facial representation/ avatar using a marker as a mental process.
As per claim 36, the claim corresponds to an insignificant extra-solution activity to the judicial exception, e.g. mere data gathering in conjunction with a law of nature or abstract idea.
As per claim 39, a human can modify/ update/ edit an image and the plurality of the patterns thereof in the form of different colors (red, green, blue, etc.) (e.g., make the hat red, green and blue with stripes using a marker or by pasting in a new hat that is different colors) using a pen/ marker and paper as a mental process.
As per claims 42- 43, the augment reality (AR) device or a smart eye glass is merely a generic display and also constitutes a standard computer interface to implement an otherwise abstract idea. The augment reality (AR) device or a smart eye glass is further routine, conventional and well- known by the virtue of the factual evidences under STEP 2- 2B (not repeated here for brevity).
As per claim 44, the claim corresponds to an insignificant extra-solution activity to the judicial exception, e.g. mere data gathering in conjunction with a law of nature or abstract idea.
As per claim 45, a human can readily identify a bar code in an image/ drawing (e.g., locate and identify a bar code on a can). The sending and/ or receiving of data corresponds to an insignificant extra-solution activity to the judicial exception, e.g. mere data gathering in conjunction with a law of nature or abstract idea.
As per claim 46, a human can readily identify a Quick Response (QR) code in an image/ drawing (e.g., locate and identify a bar code on a can). The sending and/ or receiving of data corresponds to an insignificant extra-solution activity to the judicial exception, e.g. mere data gathering in conjunction with a law of nature or abstract idea.
As per claim 47, a human can readily identify a Quick Response (QR) code and/ or a bar code in an image/ drawing (e.g., locate and identify a QR code/ bar code on a can). The sending and/ or receiving of data corresponds to an insignificant extra-solution activity to the judicial exception, e.g. mere data gathering in conjunction with a law of nature or abstract idea.
As per claim 48, a human can readily visually search/ look for and further identify similarities and/ or differences of patterns in an image/ drawing (e.g., two hats with the same and/ or different stripes and colors). The sending and/ or receiving of data corresponds to an insignificant extra-solution activity to the judicial exception, e.g. mere data gathering in conjunction with a law of nature or abstract idea.
As per claim 49, a human can readily identify a Quick Response (QR) code and/ or a bar code in an image/ drawing (e.g., locate and identify a QR code/ bar code on a can). The sending and/ or receiving of data corresponds to an insignificant extra-solution activity to the judicial exception, e.g. mere data gathering in conjunction with a law of nature or abstract idea.
Consequently, the identified additional element taken into consideration individually or in combination of the steps performed fails to amount to significantly more than the abstract idea above.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 46 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
The claims contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventors, at the time the application was filed, had possession of the claimed invention. If Applicants choose to attempt to traverse this rejection, the Examiner requests a claim chart that properly maps the corresponding disclosure to each and every limitation challenged below.
Claim 46 recites “identifying a Quick Response (QR) code, in the captured image; and wherein the sending the data associated with the recognized object comprises sending data associated with the QR code” (emphasis added).
Therefore, Applicants must show they were in “possession” of the invention at the time of filing. Notably, the specification is silent, inter alia, to the following terms: “identifying a Quick Response code… sending data associated with the QR code”.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 13, 27 and 39 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Brandit et al (US PAP 2013/ 0286048),” Brandit”.
As per claim 13, as to the broadest reasonable interpretation, Brandit teaches executing, by a computing device a pattern recognition process on a captured image to determine a recognized pattern (i.e., automatically detect the face, localize the relevant facial features (eyes, brows, nose, lips, jaw corresponding to patterns), break the portrait image into layers) see for example [0059]; [0077, 80, 100, 101, and 110- 111] disclose identification of the facial feature components broadly corresponding to the recognition process;
sending, to a remote database storing modification preferences for a plurality of patterns, data associated with the recognized pattern (i.e., instructions and data implementing desired functions wherein the Program instructions and data stored via a computer-accessible medium may be transmitted (corresponding to sending and or/ receiving) by transmission media or signals such as electrical, electromagnetic, or digital signals, which may be conveyed via a communication medium such as a network and/or a wireless link, such as may be implemented via network interface) see for example [130];
receiving, from the remote database, instructions for modifying one or more portions of the recognized pattern, wherein at least a portion of the instructions are configured to replace the one or more portions of the recognized pattern with a replacement image; and generating a modified captured image by replacing, based on the instructions, the one or more portions of the recognized pattern in the captured image with the replacement image (i.e., facial retouching are disclosed. In one embodiment, a face in an input image is detected. One or more transformation parameters (broadly corresponding to the replacement) for the detected face are estimated based on a profile model. The profile model is applied to obtain a set of feature points for each facial component of the detected face. Global and component-based shape models are applied to generate feature point locations of each facial component (corresponding to the generation of the modified image) of the detected face) see for example the abstract;
Brandit is replete with said limitation as follows:
[0029] discloses “face model consisting of seven 7 components: jawline, nose, lips, left eye, right eye, left brow, eye brow (corresponding to the patterns). An example of such a model on a face-fitting result of one embodiment is shown in FIG. 4, discussed below. In such a component-based model, feature points are obtained by transformation (corresponding to the replacement and/ or modification) from local coordinates of each component by…scale, rotation and translation of the face to the global coordinate system”.
[0050] align (corresponding to the replacement and/ or modification) current appearance with the mean shape (corresponding to the pattern) fit by finding similarity transform parameters s, R and t.sub.0 that make the best fit using equation 7
[0051] fix the similarity transform parameters, fit (corresponding to the replacement and/ or modification) appearance centroids p.sub.i to the global shape model to obtain r using equation 7 and soft nearest neighbor regularization strategy.
[0058] Some embodiments locate facial features (corresponding to the patterns) in high resolution portrait photos and automate the work flow of portrait retouching (corresponding to the replacement and/ or modification) to improve results achievable by non-experts with minimal effort. Given a portrait image, some embodiments locate the major landmarks on a face, including the eyes, nose, mouth, and face boundaries. Once the landmarks have been located, masks are constructed for the relevant face regions (eyes, lips, skin, teeth, and brows) to enable the editing step (corresponding to the replacement and/ or modification). Embodiments then present an interactive interface for the user to do common portrait corrections (corresponding to the replacement and/ or modification) such as skin softening, eye contrast and sharpening, and lip and teeth enhancement.
[0059] Some embodiments provide a semi-automatic workflow based on computer vision techniques to automatically detect the face, localize the relevant facial features (eyes, brows, nose, lips, jaw), (corresponding to the patterns) break the portrait image into layers using accurate soft masks generated from the facial feature coordinates for each of the areas of the face to be modified(corresponding to the replacement and/ or modification), and provide simple, intuitive, controls to apply the adjustments(corresponding to the replacement and/ or modification). Examples of the controls include, skin smoothness, eye sharpness and contrast, whitening of sclera and teeth, lip color and contrast adjustment, skin tone adjustment, and control of lighting on the eye area.
[0060] In addition to control over the adjustments (corresponding to the replacement and/ or modification) to the facial appearance, some embodiments allow the user to also adjust (corresponding to the replacement and/ or modification) the automatically determined feature locations, so that, for instance, if the eye corner is located incorrectly by the embodiment, the user can pull (corresponding to the replacement and/ or modification) the eye corner to the correct location, and the other point locations are moved according to this user-provided constraint.
[0061] Additionally, some embodiments learn a particular user's style preferences by saving the adjustment settings made by a user over a period of time, and choosing default settings based on these preferences. Learning users' preferences in terms of semantic adjustments (corresponding to the replacement and/ or modification).
[0103] In some embodiments, the retouch calculation module 132 performs adjusting (corresponding to the replacement and/ or modification) one or more facial component shapes (corresponding to the patterns) based on the plurality of masks. In some embodiments, the retouch calculation module 132 performs adjusting one or more facial component colors (corresponding to the patterns) based on the plurality of masks.
[0110] In some embodiments, face detection module 118 performs detecting a face in an input image 110. A feature point identification module 116 performs estimating one or more transformation parameters for the detected face based on a profile model, applying the profile model to obtain a set of feature points for each facial component of the detected face, and applying global and component-based shape models to generate feature point locations of the each facial component of the detected face. Some embodiments iterate the applying the profile model and the applying the global and component-based shape models until a change in the feature point locations is less than a threshold and present the feature point locations to a user.
[0111] In some embodiments, model management module 126 performs generating the profile model from a training image data set. In some embodiments, the estimating the transformation parameters performed by feature point identification module 116 includes calculating a translation parameter, a scale parameter and a rotation parameter (corresponding to the replacement and/ or modification) of the detected face (corresponding to the pattern) relative to the profile model. In some embodiments, feature point identification module 116 performs presenting the feature point locations, receiving user input representing corrections (corresponding to the replacement and/ or modification) to selected ones of the feature point locations, and iterating the applying the profile model and the applying the global and component-based shape models until a change in the feature point locations is less than a threshold, with respect to which the iterating the applying the profile model and the applying the global and component-based shape models further includes treating the corrections to the selected ones of the feature point locations as fixed locations for the selected ones of the feature point locations.
causing output, by one or more displays of the computing device and based on the instructions, a modified captured image (i.e., Input/output devices 1050 may, in some embodiments, include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices) see for example [133].
Note; the patterns (used for the recognition) is defined throughout the specification for the current applicant as comprising of the human body parts and ALL the characteristics/ features thereof including and not limited to the shape, size, legs, arms, lips, mouth, nose, eyes, hair, color, scar, blemish, tattoo, marking(s); audio clips and characteristics thereof including and not limited to speech, voice, sound, lisp, slipping; comprehensive object profiles; comprehensive book/ poem/ textual material/ music; IP such as trademarks, design, ETC.
As per claim 27, Brandit broadly teaches an instruction to blur at least a portion of the recognized pattern (i.e., blurring the eye/ facial component) see for example [0058 and 109].
As per claim 39, Brandit broadly teaches a first instructions to modify a first portion of the recognized pattern and a second instruction to modify a second portion of the recognized pattern (i.e., facial retouching module 120 may include a retouch (corresponding to modify/ blur/ edit) calculation module 132 for performing retouching operations (in plural corresponding to the first and second modify) on input image 110. Embodiments of facial retouching module 120 may include a model management module 126 for interacting with an image database 136 in order to refine models used for feature point identification (corresponding to feature recognition) and mask generation with respect to input image 110. Image database 136 includes images together with the annotated feature point locations, which enable the training process) see for example [100].
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 3, 8,11, 36, 42, 44, 51, 52, 53 and 55 are rejected under 35 U.S.C. 103 as being unpatentable over Sternitzke (US PAP 2013/ 0286048) in view of Wolf (US PAP 2014/ 0140575).
As per claims 2 and 51, Sternitzke teaches a CRM see for example [0058];
executing, by a computing device an object recognition process on a captured image to determine a recognized object (i.e., object/ product recognition) see for example [0034 and 43];
sending, to a remote database storing modification preferences for a plurality of objects, data associated with the recognized object (i.e., obtaining preferences stored in a database stemming from the mobile devices) see for example [0020];
receiving, from the remote database, one or more instructions for modifying one or more portions of the recognized object (i.e., the information displayed via the AR engine is adjusted to preferences stored in a database) see for example [0020, 30 and 50]; and
causing output, by one or more displays of the computing device and based on the one or more instructions, a modified captured image (i.e., the information displayed via augmented reality is adjusted to preferences stored in a user preference database and compared real time) see for example [0020, 30 and 43].
However, Sternitzke does not teach a user privacy preference indicating that at least one portion of the recognized object in captured images comprising the recognized object should be blocked from display by one or more displays of the computing device; and generating a modified captured image by modifying, based on the one or more instructions, the at least one portion of the recognized object in the captured image.
Wolf teaches a user privacy preference indicating that at least one portion of the recognized object in captured images comprising the recognized object should be blocked from display by one or more displays of the computing device; and generating a modified captured image by modifying, based on the one or more instructions, the at least one portion of the recognized object in the captured image (i.e., a display configured to receive and display the modified image data from the processor. The processor can be configured to obscure the at least a portion of the image data in response to a preference setting requesting privacy. The system can include a user interface or other device for receiving the preference setting from a user and storing the identifier of the target and the received preference setting in the storage system. The user interface can include a processor configured to transmit the identifier of the target and the received preference setting to the storage system via a network) see for example [162]; [0056] discloses “Processor 186 or receiver 130, or other components of imaging system 190, can modify image data before it is delivered, or can produce additional outputs overlaid on, displayed alongside, or delivered with the image data”.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art at the time the invention was made to incorporate the teachings of Wolf into Sternitzke wherein obscurant data can include data defining highlights or indicators as well as graphic structures, wherein a target is selected from the group consisting of a person, a building, or an animal and wherein determining of a size of the portion of the image data using the received image data, e.g., by detecting image pixels corresponding to the target using, e.g., face- or shape-recognition algorithms, is made possible and therefore the obscurant data can be used to protect privacy and further secure and safeguard undesired information/data from being made public as an state of the art technology see for example [144].
As per claims 3 and 52, Sternitzke broadly teaches searching of the remote a repository database to identify a candidate object that is similar to the recognized object (i.e., the information to be displayed as AR may either stem from database 420, or the information is generated in real-time from matching data from the database for image-object identification (410), the database with user preferences (430), the database with customer and/or product data (440) (e.g., a data warehouse), and/or the database for image-object localization) see for example [0031]; and
wherein receiving the one or more instructions comprises receiving a plurality of different modification preferences specific to the candidate object (i.e., the information displayed via the AR engine is adjusted to preferences (in plural) stored in a database stemming from the mobile devices' user (430--the user preference database), and is obtained either from the mobile devices' memory or accessible via a network interface. These preferences can be compared in real-time to preferences from third parties stored in one or multiple databases) see for example [0020, and 30].
As per claims 8 and 53, Sternitzke broadly teaches the one or more instructions are further based on a geographic location associated with the captured image or a geographic location associated with the computing device (i.e., location of the objects may be further specified by means of the objects identified via the images) see for example [0026 and 28]; [0031] discloses “the images obtained from the mobile devices' camera enable server 305 to identify the exact location of the user in the store, including the exact viewing direction using data from database 450 (im