Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This communication is a non-Final office action in merits. Claims 13-31 are presently pending. Claims 13, 18-22, and 27-31, after restriction election, have been elected and considered below.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN202111422206.X, filed on 11/26/2021.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 8/1/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Election to Restriction
Applicant elects species III, namely, claims 18-21, and 27-30, without traverse, for further examination.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or
nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 13, 18, 22, 27, 31 are rejected under 35 U.S.C. 103 as being unpatentable over US 2020/0134388 A1, Rohde (hereinafter Rohde) in view of US 2021/0150126 A1, McCulloh et al. (hereinafter McCulloh).
1.-12. (canceled)
As to claim 13, Rohde discloses a component identification method, comprising:
inputting a first image into a set identification model to obtain at least one user interface (UI) block in the first image outputted by the set identification model (Figs 19-20, 23, request to generate user interface (UI) with input data, the input data including image, and interface generator module including machine learning module (e.g. set identification model) to output user inface data; pars 0038, 0043, 0052),
wherein the first image is determined on the basis of a first visual mockup, the set identification model is used for identifying the at least one UI block in an inputted
image (Figs 3, 12, component type identification module; pars 0057, 0060-0062, 0068, 0104-0105, the same type or class of components form a component set), and each UI block at least comprises an image region obtained by UI component rendering (Fig 8, determine bounding boxes for visible elements (e.g. regions); pars 0022, 0079);
determining, among nodes of a document object model (DOM) corresponding to the first visual mockup, a first node corresponding to each UI block in the at least one
UI block (Figs 1, 4A-4B, 9B, 12B; pars 0044, 0059, 0063-0064, the leaf node(s) from DOMs received of components corresponding to the interface being identified); and
inputting an image corresponding to each first node into a set classification model to obtain a UI component type corresponding to the first node outputted by the set
classification model, the set classification model is used for determining a UI component type corresponding to an inputted image.
Rohde does not expressly disclose the input image being a mockup image and UI components/blocks generated therefore a mockup UI.
McCulloh, in the same or similar field of endeavor, further teaches identifying a mockup image being input in a UI design and a mockup UI being constructed or generated using the mockup image (Figs 1-2, 4; pars 0004-0005, 0044, 0048-0049, 0123).
Consider Rohde and McCulloh’s teachings as a whole, it would have been obvious to one of skill in the art before the filing date of invention to incorporate McCulloh’s teachings in Rohde’s method to utilize machine learning model for mockup UI design.
14-17. (Withdrawn)
As to claim 18, Rohde as modified discloses the method according to claim 13, wherein prior to inputting the first image into a set identification model, the method further comprises: training the set identification model and the set classification model based on a second visual mockup which is accordingly provided with at least one first annotation (Rohde: Figs 2A, 10, 13, providing labels according to classes or types of components; pars 0053, , 0063, 0069, 0089, 0112, 0114; McCulloh: pars 0067-0068, 0076-0077), wherein the first annotation includes first position information of an image region that can be obtained by rendering with UI components and a corresponding first label (Rohde: Fig 2A, elements being grouped based on coordinate and depth (e.g. location), classification, etc. with label element; Figs 4B, 5, 17, depth showing in image in Fig 5; pars 0102, 0112, 0134, 0159), the first position information is used for describing the position of the image region in the second visual mockup, and the first label is used for describing the type of the UI components (Rohde: Figs 2A, 4B, 5, 17; pars 0079, 0102, 0112, 0134, 0159; McCulloh: pars 0067-0068, 0076-0077).
As to claim 22, it is a device claim encompassed claim 13. Rejection of claim 13 is therefore incorporated herein.
23-26. (Withdrawn)
As to claim 27, it is rejected with the same reason as set forth in claim 18.
As to claim 31, it recites a storage medium storing computer program executed to perform steps of claim 13. Rejection of claim 13 is therefore incorporated herein.
Allowable Subject Matter
Claims 19-21, 28-30 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Reasons for Allowance
Prior art of record (Rohde and McCulloh) neither discloses alone nor teaches in combination functions and features recited in claim 19. Claim 28 recites similar limitations as claim 19. Claims 20-21 and 29-30 depend from claims 19 and 28, respectively.
Examiner’s Note
Examiner has cited particular column, line number, paragraphs and/or figure(s) in the reference(s) as applied to the claims for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the reference(s) in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Qun Shen whose telephone number is (571) 270-7927. The examiner can normally be reached on Mon-Friday from 9:00-5:00. If attempts to reach the examiner by telephone are unsuccessful, the examiner's Supervisor, Amandeep Saini can be reached on (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/QUN SHEN/
Primary Examiner, Art Unit 2662
Prosecution Insights