DETAILED ACTION
Status of the Application
1 Applicant’s Amendment to the Claims filed January 16, 2026 are received and entered.
2. Claims 2 and 8 are cancelled. Claims 1, 12, and 13 are amended. Claims 1 – 13 are pending and are under examination in this action.
3. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments / Amendment
4. On pages 8 – 9 of the Response, Applicant argues that the prior art fails to teach that when the “portion” is “behind an object, displaying the first visual representation of the input having a first line style” and when the “portion” is not “behind the object, displaying the first visual representation of the input having a second line style that is different from the first line style”, as newly added to claims 1, 12, and 13.
Applicant’s arguments have been fully considered and are persuasive in view of the newly added subject matter. However, upon further consideration, a new ground(s) of rejection is made in view of Goda (U.S. Pub. 2014/0165011).
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1, 9, and 12 – 13 are rejected under 35 U.S.C. 103 as being unpatentable over Jung et al. (U.S. Pub. 2016/0097651) in view of Goda (U.S. Pub. 2014/0165011).
Regarding claim 1, Jung teaches: a computer system configured to communicate with one or more display generation components and one or more input devices (FIGS. 1, 8B; paragraphs [0053], [0067], [0068]; device 100 [computer system] includes display unit 151 and sensing unit 140 that detects touch inputs applied to the display unit 151), comprising:
one or more processors (FIG. 1; paragraph [0092]; controller 180); and
memory storing one or more programs configured to be executed by the one or more processors (FIG. 1; paragraphs [0089], [0095]; programs may be stored in memory 160 and executed by controller 180 to perform disclosed functions), the one or more programs including instructions for:
while displaying, via the one or more display generation components, a representation of screen content that is being shared from an external computer system while the computer system does not have control of the screen content that is being shared from the external computer system (FIG. 8B; paragraph [0204]; device 100 [computer system] displays a first map screen [content] via display unit 151. This “content” is shared from display apparatus 200 [external computer system]. Since this “content” originates at display apparatus 200 [external computer system], device 100 [computer system] does not have control of the “content”):
detecting, via the one or more input devices, an input directed to the representation of the screen content (FIG. 8B; paragraph [0204]; user inputs of drawing shapes or lines on the first map screen [content], such as an “X” as illustrated, are detected by sensing unit 140 of device 100 [computer system]), wherein the input directed to the representation of the screen content includes a drawing gesture that includes movement of the input relative to the displayed representation of the screen content (FIG. 8B; paragraph [0204]; as set forth above, the inputs include drawing shapes or lines such as an “X” that includes movement of a user’s finger as the shapes / lines are being drawn over the displayed content); and
in response to detecting the input directed to the representation of the screen content:
displaying, via the one or more display generation components, a first visual representation of the input, wherein the first visual representation of the input is displayed over at least a portion of the representation of the screen content (FIG. 8B; paragraph [0204] – [0207]; a drawing input, such as a first line 820a in the shape of an “X” [first visual representation] is displayed via display unit 151 of device 100 [computer system]. The first line 820a [first visual representation] is displayed over the first map screen [content] and has a first color, first thickness, etc.); and
initiating a process for causing a second visual representation of the input to be displayed over the screen content at the external computer system (FIG. 8B; paragraph [0204] – [0207]; a drawing input, such as a second line 820b in the shape of an “X” [second visual representation] is displayed by display apparatus 200 [external computer system]. The second line 820b [second visual representation] is displayed over the first map screen [content] and has a second color, second thickness, etc.).
Jung fails to explicitly disclose: wherein displaying the first visual representation of the input over at least the portion of the representation of the screen content includes: in accordance with a determination that the portion of the representation of the screen content is positioned behind an object, displaying the first visual representation of the input having a first line style; and in accordance with a determination that the portion of the representation of the screen content is not positioned behind the object, displaying the first visual representation of the input having a second line style that is different from the first line style.
However, Jung discloses various interfaces or objects or icons (FIGS. 10B, 13; 1030, 1040, 1211, 1212, 1213, 1214, 1215, 1216) that may be displayed in conjunction with the “first visual representation” of an X, or other drawn user input such as line 1020a. When the icons of FIG. 13 are displayed in conjunction with drawn user input, such as line 1020a, there may be some amount of overlap. Accordingly, Jung suggests that the “first visual representation of the input” may be overlapped by some interfaces, objects, or icons.
With regard to claim 1, Goda teaches: wherein displaying the first visual representation of the input over at least the portion of the representation of the screen content includes: in accordance with a determination that the portion of the representation of the screen content is positioned behind an object, displaying the first visual representation of the input having a first line style (FIG. 22B; paragraph [0196]; when content, such as image B [representation], is positioned behind image C [object], image B [representation] is illustrated by dotted lines [first line style] to visually show the overlapping); and
in accordance with a determination that the portion of the representation of the screen content is not positioned behind the object, displaying the first visual representation of the input having a second line style that is different from the first line style (FIG. 22B; paragraph [0196]; when content, such as image E [representation], is not positioned behind image C [object], image E [representation] is illustrated by solid lines [second line style] to visually show no overlapping).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung and Goda to yield predictable results. More specifically, the teachings of a system that displays content that may overlap, as taught by Jung, are known. Additionally, the teachings of a display device which displays overlapped content with a line style different from content that does not overlap, as taught by Goda, are known. The combination of the known teachings of Jung and Goda would yield the predictable result of a system that displays content that may overlap, such that overlapped content is displayed with a line style different from content that does not overlap. More specifically, it would have been obvious to simply applying the different line styles of Goda with regard to overlapping and non-overlapping content of Jung to visually distinguish overlapping and non-overlapping content. Such a combination merely constitutes an implementation of an aesthetic design choice to visually distinguish overlapped and non-overlapped content. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung and Goda to yield the aforementioned predictable results.
Additionally, please see MPEP §2144.04(I) which refers to case law that has held that “matters relating to ornamentation only which have no mechanical function cannot be relied upon to patentably distinguish the claimed invention from the prior art.”
In this case, the only difference between Jung and the prior art is the aesthetic choice of how overlapping displayed content is displayed compared to non-overlapping displayed content. The particular “line styles” recited by Applicant amount to nothing more than an aesthetic design choice having no bearing on the functionality or operation of the claimed device. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s claimed invention to modify the teachings of Jung to have a different aesthetic design when content overlaps, in view of established case law.
Regarding claim 9, Jung teaches: wherein displaying the first visual representation of the input includes: in accordance with a determination that the input includes a first type of input, displaying the first visual representation of the input having a first set of one or more visual effects (FIG. 8B; paragraph [0204], [0207]; when the user’s input has a first shape of an “X} [first type], first line 820a [first visual representation] is displayed with a first color, first thickness, etc. [first set of visual effects]).
Regarding claim 12, Jung teaches: a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors, the one or more programs including instructions for performing operations (FIG. 1; paragraphs [0089], [0095]; programs may be stored in memory 160 [computer-readable medium] and executed by controller 180 to perform disclosed functions).
The remainder of this claim was rejected above with regard to claim 1. Accordingly, the remainder of this claim is rejected for at least the same reasons as set forth above with regard to claim 1. A duplication of the above rejection is not included in this Office Action for the purpose of brevity.
Regarding claim 13, this claim is a method recitation of the functional operations rejected above with regard to claim 1. Accordingly, this claim is rejected for at least the same reasons as set forth above with regard to claim 1. A duplication of the above rejection is not included in this Office Action for the purpose of brevity.
7. Claims 3 – 4, 7, and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Jung in view of Goda, as applied to claims 1 and 9 above, in further view of Chaudhri et al. (U.S. Pub. 2017/0357324).
Regarding claim 3, neither Jung nor Goda explicitly disclose: wherein displaying the first visual representation of the input includes: displaying the first visual representation of the input for a predetermined amount of time; and after displaying the first visual representation of the input for the predetermined amount of time, ceasing display of the first visual representation of the input.
However, in a related field of endeavor, Chaudhri discloses: a device 600 that receives touch inputs 608 and outputs visual representations thereof that may include free-form line drawings (FIG. 6A; paragraph [0215]).
With regard to claim 3, Chaudhri teaches: wherein displaying the first visual representation of the input includes: displaying the first visual representation of the input for a predetermined amount of time (FIG. 6C; paragraph [0220]; the displayed visual representations of user input are displayed for a predetermined period of time); and
after displaying the first visual representation of the input for the predetermined amount of time, ceasing display of the first visual representation of the input (FIG. 6C; paragraph [0220]; after the predetermined period of time has elapsed, the displayed visual representations of user input fade-out).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung, Goda, and Chaudhri to yield predictable results. More specifically, the teachings of a touch display device which receives and displays drawing inputs, as taught by Jung, are known. Additionally, the teachings of a touch display device which receives and displays drawing inputs for a predetermined period of time after which the displayed drawing inputs fade-out, as taught by Chaudhri, are known. The combination of the known teachings of Jung and Chaudhri would yield the predictable result of a touch display device which receives and displays drawing inputs for a predetermined period of time after which the displayed drawing inputs fade-out. More specifically, it would have been obvious to simply add the fade-out animation after a predetermined period of time has elapsed with regard to drawn inputs, as taught by Chaudhri, to the drawn inputs of Jung. Such a combination merely incorporates the known fade-out of drawn inputs after a predetermined time has elapsed, as taught by Chaudhri, to the similar device of Jung which also receives drawing inputs from a user. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung, Goda, and Chaudhri to yield the aforementioned predictable results.
Regarding claim 4, neither Jung nor Goda explicitly disclose: wherein ceasing display of the first visual representation of the input includes displaying an animated effect in which the first visual representation of the input fades away over a period of time.
However, Chaudhri teaches: wherein ceasing display of the first visual representation of the input includes displaying an animated effect in which the first visual representation of the input fades away over a period of time (FIG. 6C; paragraph [0220]; the displayed visual representations of user input are displayed for a predetermined period of time. After the predetermined period of time has elapsed, the displayed visual representations of user input are displayed with a fade-out animation).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung, Goda, and Chaudhri to yield predictable results for at least the reasons set forth above with regard to claim 3.
Regarding claim 7, neither Jung nor Goda explicitly disclose: wherein displaying the first visual representation of the input includes animating the representation of the input over time.
However, Chaudhri teaches: wherein displaying the first visual representation of the input includes animating the representation of the input over time (FIG. 6C; paragraph [0220]; the displayed visual representations of user input are displayed for a predetermined period of time. After the predetermined period of time has elapsed, the displayed visual representations of user input are displayed with a fade-out animation).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung, Goda, and Chaudhri to yield predictable results for at least the reasons set forth above with regard to claim 3.
Regarding claim 10, neither Jung nor Goda explicitly disclose: wherein displaying the first visual representation of the input having the first set of one or more visual effects includes: while the input is detected at the computer system, continuing to display the first visual representation of the input having at least one visual effect from the first set of one or more visual effects; and in response to detecting that the input is terminated, ceasing display of the first visual representation of the input.
However, Chaudhri teaches: wherein displaying the first visual representation of the input having the first set of one or more visual effects includes: while the input is detected at the computer system, continuing to display the first visual representation of the input having at least one visual effect from the first set of one or more visual effects (FIG. 6C; paragraph [0220]; the displayed visual representations of user input are displayed for a predetermined period of time starting from when the user applies the user input); and
in response to detecting that the input is terminated, ceasing display of the first visual representation of the input (FIG. 6C; paragraph [0220]; after the user’s input has finished and a predetermined period of time has elapsed, the displayed visual representations of user input fade-out, i.e. the display of the visual representations of user input ends).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung, Goda, and Chaudhri to yield predictable results for at least the reasons set forth above with regard to claim 3.
8. Claims 5 – 6 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Jung in view of Goda, as applied to claims 1 and 9 above, in further view of Stern et al. (U.S. Pub. 2021/0064143).
Regarding claim 5, neither Jung nor Goda explicitly disclose: wherein displaying the first visual representation of the input includes displaying the first visual representation of the input having a set of one or more colors that are determined based on one or more visual properties of the portion of the representation of the screen content.
However, Stern teaches: wherein displaying the first visual representation of the input includes displaying the first visual representation of the input having a set of one or more colors that are determined based on one or more visual properties of the portion of the representation of the screen content (paragraph [0046]; the color of a displayed object, in this case feedback indicator 120 [first visual representation], is automatically adjusted / changed so as to not match [visual properties] the background color [the portion of the representation of the screen content]. This ensures that the displayed object [feedback indicator 120] is human-discernable from the background color by having sufficient contrast).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung, Goda, and Stern to yield predictable results. More specifically, the teachings of a touch display device which displays drawing inputs having color, as taught by Jung, are known. Additionally, the teachings of automatically changing the color(s) of displayed objects to contrast with a background color in a human-discernable manner, as taught by Stern , are known. The combination of the known teachings of Jung and Stern would yield the predictable result of a touch display device which displays drawing inputs having at least one color, where the at least one color is automatically provided to contrast with a background color in a human-discernable manner. More specifically, it would have been obvious to simply display the “first visual representation” of Jung with a color that easily contrasts the portion of the first map screen [content] that is being overlapped, as taught by Stern. Such a combination merely incorporates the benefit of human-discernable contrast to ensure that the “first visual representation” of Jung is always visible to a user. This would apply to multiple colors of the “first visual representation” as necessitated by multiple colors of the portion of the first map screen [content / background], as appropriate. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung, Goda, and Stern to yield the aforementioned predictable results.
Additionally, please see MPEP §2144.04(I) which refers to case law that has held that “matters relating to ornamentation only which have no mechanical function cannot be relied upon to patentably distinguish the claimed invention from the prior art.”
In this case, the only difference between Jung and the prior art is the aesthetic choice of how overlapping displayed content is displayed to visually distinguish said content. The particular “colors” recited by Applicant amount to nothing more than an aesthetic design choice having no bearing on the functionality or operation of the claimed device. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s claimed invention to modify the teachings of Jung to have a different aesthetic design when content overlaps, in view of established case law.
Regarding claim 6, neither Jung nor Goda explicitly disclose: wherein displaying the first visual representation of the input includes: displaying a first portion of the first visual representation of the input over a first portion of the representation of the screen content, wherein the first portion of the first visual representation of the input includes a first color and the first portion of the representation of the screen content includes a second color that is different from the first color; and displaying a second portion of the first visual representation of the input over a second portion of the representation of the screen content, including: in accordance with a determination that the second portion of the representation of the screen content has a third color, displaying the second portion of the first visual representation of the input having a fourth color that is different from the first color; and in accordance with a determination that the second portion of the representation of the screen content has a fifth color that is different from the third color, displaying the second portion of the first visual representation of the input having a sixth color that is different from the fourth color.
However, Stern teaches: wherein displaying the first visual representation of the input includes: displaying a first portion of the first visual representation of the input over a first portion of the representation of the screen content, wherein the first portion of the first visual representation of the input includes a first color and the first portion of the representation of the screen content includes a second color that is different from the first color (paragraph [0046]; the color of a displayed object, in this case feedback indicator 120 [first visual representation], is automatically adjusted / changed so as to not match [visual properties] the background color [the portion of the representation of the screen content]. This ensures that the displayed object [feedback indicator 120] is human-discernable from the background color by having sufficient contrast. The object would have a first color and the first portion of the background disposed beneath would have a second color); and
displaying a second portion of the first visual representation of the input over a second portion of the representation of the screen content, including:
in accordance with a determination that the second portion of the representation of the screen content has a third color, displaying the second portion of the first visual representation of the input having a fourth color that is different from the first color (paragraph [0046]; when the object is also displayed over a second portion of the background having a third color, the object would have a fourth color different from the third color to have sufficient contrast with the third color); and
in accordance with a determination that the second portion of the representation of the screen content has a fifth color that is different from the third color, displaying the second portion of the first visual representation of the input having a sixth color that is different from the fourth color (paragraph [0046]; when the object is also displayed over a second portion of the background having a fifth color, the object would have a sixth color different from the fourth color to have sufficient contrast with the fifth color).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung, Goda, and Stern to yield predictable results for at least the reasons set forth above with regard to claim 5.
Regarding claim 11, Jung teaches: the one or more programs further including instructions for: while the input is detected at the computer system and the first visual representation of the input is displayed having a first appearance that includes the first set of one or more visual effects, detecting, via the one or more input devices, movement of the input (FIG. 8B; paragraphs [0204], [0206]; as set forth above, user inputs of drawing shapes or lines on the first map screen [content]. This includes movement of a user’s finger as the shapes / lines are being drawn over the displayed content. The first line 820a [first visual representation] is displayed with a first color [first set of visual effects]).
Neither Jung nor Goda explicitly disclose: in response to detecting the movement of the input, displaying, via the one or more display generation components, the first visual representation of the input having a second appearance that is different from the first appearance, wherein the second appearance includes a second set of one or more visual effects that is different from the first set of one or more visual effects.
However, Stern teaches: in response to detecting the movement of the input, displaying, via the one or more display generation components, the first visual representation of the input having a second appearance that is different from the first appearance, wherein the second appearance includes a second set of one or more visual effects that is different from the first set of one or more visual effects (paragraph [0046]; the color of a displayed object, in this case feedback indicator 120 [first visual representation], is automatically adjusted / changed so as to not match [visual properties] the background color [the portion of the representation of the screen content]. This ensures that the displayed object [feedback indicator 120] is human-discernable from the background color by having sufficient contrast. The “first visual representation” would have a first color [first visual effect] when provided over a first portion of the background having a second color. As the “first visual representation” is continued applied by a user drawing [see Jung above] over a second portion of the background having a third color, the color of the “first visual representation may automatically change to a fourth color [second visual effect]).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of Applicant’s failed invention to combine the known teachings of Jung, Goda, and Stern to yield predictable results for at least the reasons set forth above with regard to claim 5.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN A LUBIT whose telephone number is (571)270-3389. The examiner can normally be reached M - F, ~6am - 3pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Temesghen Ghebretinsae can be reached at 571-272-3017. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN A LUBIT/Primary Examiner, Art Unit 2626